{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T10:23:19Z","timestamp":1790936599429,"version":"4.1.0"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"5","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62302347 and U2436205"],"award-info":[{"award-number":["62302347 and U2436205"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>GitHub actions (GHA), a built-in continuous integration and continuous delivery (CI\/CD) service of GitHub, has been widely adopted by developers, streamlining the automation of software development workflows. Despite its popularity, failures frequently occur during GHA workflow executions. Fixing these failures often requires significant human effort, and unsuccessful workflow executions waste computing resources. Understanding the reasons behind workflow failures could provide valuable insights for troubleshooting the existing issues of CI\/CD and further improving the development process.<\/jats:p>\n                  <jats:p>In this article, we present an empirical study to reveal the reasons behind GHA workflow failures. By manually analyzing 375 failed workflow executions across 260 open-source Java projects, we built a comprehensive taxonomy categorizing the common failure types. The taxonomy was further validated by surveying 151 developers. This study is the first empirical work to analyze GHA workflow failures, bringing valuable knowledge to the field of continuous integration in software engineering. Moreover, our taxonomy and survey results not only underscore the critical need for better tools and practices to mitigate these failures but also indicate the directions to enhance the efficiency and reliability of CI\/CD pipelines.<\/jats:p>","DOI":"10.1145\/3749371","type":"journal-article","created":{"date-parts":[[2025,7,18]],"date-time":"2025-07-18T14:24:19Z","timestamp":1752848659000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Why Do GitHub Actions Workflows Fail? An Empirical Study"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5872-1616","authenticated-orcid":false,"given":"Lianyu","family":"Zheng","sequence":"first","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8865-9267","authenticated-orcid":false,"given":"Shuang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-6955-0969","authenticated-orcid":false,"given":"Xi","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1585-1185","authenticated-orcid":false,"given":"Jiangnan","family":"Huang","sequence":"additional","affiliation":[{"name":"Radboud Universiteit, Nijmegen, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6307-8460","authenticated-orcid":false,"given":"Bin","family":"Lin","sequence":"additional","affiliation":[{"name":"Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7410-9146","authenticated-orcid":false,"given":"Jinfu","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2968-3496","authenticated-orcid":false,"given":"Jifeng","family":"Xuan","sequence":"additional","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,24]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Pekka Abrahamsson Outi Salo Jussi Ronkainen and Juhani Warsta. 2017. Agile software development methods: Review and analysis. arXiv:1709.08439. Retrieved from https:\/\/arxiv.org\/abs\/1709.08439"},{"key":"e_1_3_2_3_2","unstructured":"Amazon Ion. 2023. Ion Java GitHub Actions Run #6829246558 Job #18575023797. Retrieved June 8 2024 from https:\/\/github.com\/amazon-ion\/ion-java\/actions\/runs\/6829246558\/job\/18575023797"},{"key":"e_1_3_2_4_2","unstructured":"Andrey Novikov. 2023. Trekarta GitHub Actions Run #6631013238 Job #18013616500. Retrieved June 8 2024 from https:\/\/github.com\/andreynovikov\/trekarta\/actions\/runs\/6631013238\/job\/18013616500"},{"key":"e_1_3_2_5_2","unstructured":"Apache. 2023. Dubbo GitHub Actions Run #6835753665 Job #18589976238. Retrieved June 8 2024 from https:\/\/github.com\/apache\/dubbo\/actions\/runs\/6835753665\/job\/18589976238"},{"key":"e_1_3_2_6_2","unstructured":"aws-containers. 2023. aws-containers\/retail-store-sample-app GitHub Actions Run #6723607295 Job #18274043718. Retrieved June 8 2024 from https:\/\/github.com\/aws-containers\/retail-store-sample-app\/actions\/runs\/6723607295\/job\/18274043718"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.62"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/800253.807736"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623303"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644867"},{"key":"e_1_3_2_11_2","unstructured":"Casid. 2023. JTE GitHub Actions Run #6749461418 Job #18349849068. Retrieved June 8 2024 from https:\/\/github.com\/casid\/jte\/actions\/runs\/6749461418\/job\/18349849068"},{"key":"e_1_3_2_12_2","unstructured":"CC-Tweaked. 2023. CC-Tweaked GitHub Actions Run #6725853717 Job #18281028796. Retrieved June 8 2024 from https:\/\/github.com\/cc-tweaked\/CC-Tweaked\/actions\/runs\/6725853717\/job\/18281028796"},{"key":"e_1_3_2_13_2","article-title":"T-evos: A large-scale longitudinal study on ci test execution and failure","author":"Chen An Ran","year":"2022","unstructured":"An Ran Chen, Tse-Hsun Peter Chen, and Shaowei Wang. 2022. T-evos: A large-scale longitudinal study on ci test execution and failure. IEEE Transactions on Software Engineering 49, 4 (2022), 2352--2365.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_14_2","unstructured":"Crazy Marvin. 2023. Morse GitHub Actions Run #6672595913 Job #18136820559. Retrieved June 8 2024 from https:\/\/github.com\/crazy-marvin\/morse\/actions\/runs\/6672595913\/job\/18136820559"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00074"},{"key":"e_1_3_2_16_2","volume-title":"Biostatistics: A Foundation for Analysis in the Health Sciences","author":"Daniel Wayne W.","year":"2018","unstructured":"Wayne W. Daniel and Chad L. Cross. 2018. Biostatistics: A Foundation for Analysis in the Health Sciences. Wiley."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111827"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380351"},{"key":"e_1_3_2_19_2","volume-title":"Continuous Integration: improving Software Quality and Reducing Risk","author":"Duvall Paul M.","year":"2007","unstructured":"Paul M. Duvall, Steve Matyas, and Andrew Glover. 2007. Continuous Integration: improving Software Quality and Reducing Risk. Pearson Education."},{"key":"e_1_3_2_20_2","unstructured":"Martin Fowler and Matthew Foemmel. 2006. Continuous integration. Retrieved from http:\/\/martinfowler. com\/articles\/continuousIntegration.html"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1177\/1098214011426594"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238171"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09695-9"},{"key":"e_1_3_2_24_2","unstructured":"Inc. GitHub. 2024. Storing and sharing data from a workflow. Retrieved June 8 2024 from https:\/\/docs.github.com\/en\/actions\/writing-workflows\/choosing-what-your-workflow-does\/storing-and-sharing-data-from-a-workflow"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00084"},{"key":"e_1_3_2_26_2","unstructured":"Google Maps Samples. 2023. Android Samples GitHub Actions Run #6850982659 Job #18626307968. Retrieved from June 8 2024 https:\/\/github.com\/googlemaps-samples\/android-samples\/actions\/runs\/6850982659\/job\/18626307968"},{"key":"e_1_3_2_27_2","volume-title":"Unit Test Frameworks: Tools for High-Quality Software Development","author":"Hamill Paul","year":"2004","unstructured":"Paul Hamill. 2004. Unit Test Frameworks: Tools for High-Quality Software Development. O\u2019Reilly Media, Inc."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.21037\/jtd.2017.09.14"},{"key":"e_1_3_2_29_2","volume-title":"Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation","author":"Humble Jez","year":"2010","unstructured":"Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Pearson Education."},{"key":"e_1_3_2_30_2","unstructured":"Pamela Hunter. 2016. Margin of error and confidence levels made simple. Retrieved February 25 2016 https:\/\/psy311.wichita.edu\/wp-content\/uploads\/2020\/07\/Margin-of-error-made-simple.pdf."},{"key":"e_1_3_2_31_2","unstructured":"Hyperledger Fabric. 2023. Fabric Gateway Java GitHub Actions Run #6319347700 Job #17160112108. Retrieved from June 8 2024 from https:\/\/github.com\/hyperledger\/fabric-gateway-java\/actions\/runs\/6319347700\/job\/17160112108"},{"key":"e_1_3_2_32_2","unstructured":"Idempiere. 2023. Idempiere GitHub Actions Run #6811617984 Job #18522322951. Retrieved from June 8 2024 https:\/\/github.com\/idempiere\/idempiere\/actions\/runs\/6811617984\/job\/18522322951"},{"key":"e_1_3_2_33_2","unstructured":"Jenkins CI. 2023. GitLab Plugin GitHub Actions Run #6739653239 Job #18321512657. Retrieved from June 8 2024 https:\/\/github.com\/jenkinsci\/Gitlab-plugin\/actions\/runs\/6739653239\/job\/18321512657"},{"key":"e_1_3_2_34_2","unstructured":"JetBrains. 2023. JetBrains Runtime GitHub Actions Run #6827200507 Job #18568801104. Retrieved from June 8 2024 https:\/\/github.com\/jetbrains\/jetbrainsruntime\/actions\/runs\/6827200507\/job\/18568801104"},{"key":"e_1_3_2_35_2","volume-title":"Metrics and Models in Software Quality Engineering","author":"Kan Stephen H.","year":"2003","unstructured":"Stephen H. Kan. 2003. Metrics and Models in Software Quality Engineering. Addison-Wesley."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.26"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00054"},{"key":"e_1_3_2_38_2","first-page":"2747","volume-title":"Proceedings of the 31st USENIX Security Symposium (USENIX Security \u201922)","author":"Koishybayev Igibek","year":"2022","unstructured":"Igibek Koishybayev, Aleksandr Nahapetyan, Raima Zachariah, Siddharth Muralee, Bradley Reaves, Alexandros Kapravelos, and Aaravind Machiry. 2022. Characterizing the security of GitHub \\(\\{\\) CI \\(\\}\\) workflows. In Proceedings of the 31st USENIX Security Symposium (USENIX Security \u201922), 2747\u20132763."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1990.131377"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409760"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623351"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2012.031"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611668"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/2076450.2076466"},{"key":"e_1_3_2_45_2","unstructured":"Openmf. 2023. Openmf GitHub Actions Run #6767425387 Job #18390040404. Retrieved June 8 2024 from https:\/\/github.com\/openmf\/mobile-wallet\/actions\/runs\/6767425387\/job\/18390040404"},{"key":"e_1_3_2_46_2","unstructured":"Pig Mesh. 2023. Pig Mesh GitHub Actions Run #6622836485 Job #17988915393. Retrieved June 8 2024 from https:\/\/github.com\/pig-mesh\/pig\/actions\/runs\/6622836485\/job\/17988915393"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2637"},{"key":"e_1_3_2_48_2","unstructured":"Pulumi. 2023. Pulumi EKS GitHub Actions Run #6821578644 Job #18552321847. Retrieved June 8 2024 from https:\/\/github.com\/pulumi\/pulumi-eks\/actions\/runs\/6821578644\/job\/18552321847"},{"key":"e_1_3_2_49_2","unstructured":"Quarkus IO. 2023. Quarkus Images GitHub Actions Run #6520792390 Job #17708710967. Retrieved June 8 2024 from https:\/\/github.com\/quarkusio\/quarkus-images\/actions\/runs\/6520792390\/job\/17708710967"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.54"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/0096-0551(78)90041-3"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10285-5"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2006.91"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106392"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-021-00319-5"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3593434.3593475"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2685629"},{"key":"e_1_3_2_58_2","volume-title":"Card Sorting: Designing Usable Categories","author":"Spencer Donna","year":"2009","unstructured":"Donna Spencer. 2009. Card Sorting: Designing Usable Categories. Rosenfeld Media."},{"key":"e_1_3_2_59_2","unstructured":"Spotify. 2023. Missing Link GitHub Actions Run #6296482729 Job #17091557253. Retrieved June 8 2024 from https:\/\/github.com\/spotify\/missinglink\/actions\/runs\/6296482729\/job\/17091557253"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.2316\/P.2013.796-012"},{"key":"e_1_3_2_61_2","first-page":"1","article-title":"Acceptance and use of ChatGPT in the academic community","author":"Strzelecki Artur","year":"2024","unstructured":"Artur Strzelecki, Karina Cicha, Mariia Rizun, and Paulina Rotek. 2024. Acceptance and use of ChatGPT in the academic community. Education and Information Technologies, 1\u201326.","journal-title":"Education and Information Technologies"},{"key":"e_1_3_2_62_2","unstructured":"TIOBE Software. 2023. TIOBE Index\u2014The Software Quality Company. Retrieved November 23 2023 from https:\/\/www.tiobe.com\/tiobe-index\/"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.67"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS-C.2017.59"},{"key":"e_1_3_2_65_2","unstructured":"Zalando. 2023. Logbook GitHub Actions Run #6837892949 Job #18594402544. Retrieved from June 8 2024 https:\/\/github.com\/zalando\/logbook\/actions\/runs\/6837892949\/job\/18594402544"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09687-9"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338917"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/267580.267590"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00015"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2017.13"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749371","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T06:33:00Z","timestamp":1777098780000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749371"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,24]]},"references-count":69,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3749371"],"URL":"https:\/\/doi.org\/10.1145\/3749371","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,24]]},"assertion":[{"value":"2024-11-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}