{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T05:00:44Z","timestamp":1781154044438,"version":"3.54.1"},"reference-count":48,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2026,6,7]],"date-time":"2026-06-07T00:00:00Z","timestamp":1780790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Continuous Integration pipelines rely on large-scale automated testing to support rapid releases. However, flaky tests exhibit non-deterministic outcomes under an identical code and configuration, substantially increasing rerun costs and hindering fault localization. Existing approaches struggle to uniformly model heterogeneous runtime evidence and its multi-relational structure in CI environments, which limits cross-project generalization and interpretability. To address this gap, this paper presents HgtFlaky, a runtime-evidence-centered multi-view heterogeneous graph learning framework. A Unified Event Model is introduced to normalize heterogeneous CI artifacts into semantically consistent event quadruples, and a heterogeneous execution graph is then constructed to capture testing entities and multiple relation types. Based on the HEG, three complementary views are derived to characterize run-level, test-level, and thread-level flaky behaviors. A heterogeneous graph Transformer is further adopted to jointly encode the multi-view graph instances and learn transferable test-level representations for flaky\/non-flaky prediction. Experiments on two benchmark datasets, FlakeFlagger and IDoFT, show that HgtFlaky achieves strong and stable performance. Under 10-fold cross-validation, it obtains an F1-score of 83% on FlakeFlagger and 98% on IDoFT. Under per-project validation on FlakeFlagger, HgtFlaky achieves 78% Precision, 89% Recall, and 81% F1-score, outperforming Flakify by 8 percentage points and FlakeFlagger by 74 percentage points in F1-score.<\/jats:p>","DOI":"10.3390\/computers15060372","type":"journal-article","created":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T05:43:46Z","timestamp":1780897426000},"page":"372","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Heterogeneous Graph Transformer with Multi-View Representation Learning for Flaky Test Detection"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1919-2498","authenticated-orcid":false,"given":"Peng","family":"Dai","sequence":"first","affiliation":[{"name":"School of Big Data and Artificial Intelligence, Chizhou University, Chizhou 247100, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9428-3505","authenticated-orcid":false,"given":"Xiaoqin","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Big Data and Artificial Intelligence, Chizhou University, Chizhou 247100, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanyang","family":"Zhao","sequence":"additional","affiliation":[{"name":"BNRist, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunzhan","family":"Gong","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/TSE.2022.3140868","article-title":"The secret life of software vulnerabilities: A large-scale empirical study","volume":"49","author":"Iannone","year":"2022","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_2","unstructured":"Krasner, H. (2026, May 01). The Cost of Poor Software Quality in the US: A 2020 Report. Consortium for Information & Software QualityTM. Available online: https:\/\/www.it-cisq.org\/cisq-files\/pdf\/CPSQ-2020-report.pdf."},{"key":"ref_3","first-page":"1","article-title":"Continuous integration and deployment strategies for streamlined DevOps in software engineering and application delivery","volume":"14","author":"Ugwueze","year":"2025","journal-title":"Int. J. Comput. Appl. Technol. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3428270","article-title":"A large-scale longitudinal study of flaky tests","volume":"4","author":"Lam","year":"2020","journal-title":"Proc. ACM Program. Lang."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3476105","article-title":"A survey of flaky tests","volume":"31","author":"Parry","year":"2021","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1007\/s10664-023-10307-w","article-title":"Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models","volume":"28","author":"Parry","year":"2023","journal-title":"Empir. Softw. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lam, W., Godefroid, P., Nath, S., Santhiar, A., and Thummalapenta, S. (2019, January 15\u201319). Root causing flaky tests in a large-scale industrial setting. Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, Beijing, China.","DOI":"10.1145\/3293882.3330570"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Shabani, T., Nashid, N., Alian, P., and Mesbah, A. (2024). Dockerfile Flakiness: Characterization and Repair. arXiv.","DOI":"10.1109\/ICSE55347.2025.00238"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Akli, A., Haben, G., Habchi, S., Papadakis, M., and Le Traon, Y. (2023). FlakyCat: Predicting flaky tests categories using few-shot learning. Proceedings of the 2023 IEEE\/ACM International Conference on Automation of Software Test (AST), IEEE.","DOI":"10.1109\/AST58925.2023.00018"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"More, R., and Bradbury, J.S. (2025). An Analysis of LLM Fine-Tuning and Few-Shot Learning for Flaky Test Detection and Classification. Proceedings of the 2025 IEEE Conference on Software Testing, Verification and Validation (ICST), IEEE.","DOI":"10.1109\/ICST62969.2025.10988995"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2039","DOI":"10.1109\/TSE.2022.3208864","article-title":"Test flakiness across programming languages","volume":"49","author":"Barbosa","year":"2022","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"An, G., Yoon, J., Bach, T., Hong, J., and Yoo, S. (2024). Just-in-time flaky test detection via abstracted failure symptom matching. Proceedings of the 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE.","DOI":"10.1109\/ICSME58944.2024.00078"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gruber, M., Heine, M., Oster, N., Philippsen, M., and Fraser, G. (2023). Practical flaky test prediction using common code evolution and test history data. Proceedings of the 2023 IEEE Conference on Software Testing, Verification and Validation (ICST), IEEE.","DOI":"10.1109\/ICST57152.2023.00028"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lam, W., Shi, A., Oei, R., Zhang, S., Ernst, M.D., and Xie, T. (2020). Dependent-test-aware regression testing techniques. Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery.","DOI":"10.1145\/3395363.3397364"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ahmad, A., Held, E.N., Leifler, O., and Sandahl, K. (2022). Identifying randomness related flaky tests through divergence and execution tracing. Proceedings of the 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), IEEE Computer Society.","DOI":"10.1109\/ICSTW55395.2022.00057"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3146","DOI":"10.1109\/TSE.2024.3472476","article-title":"FlakyFix: Using large language models for predicting flaky test fix categories and test code repair","volume":"50","author":"Fatima","year":"2024","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Parry, O., Kapfhammer, G.M., Hilton, M., and McMinn, P. (2022, January 21\u201329). Surveying the developer experience of flaky tests. Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice, Pittsburgh, PA, USA.","DOI":"10.1145\/3510457.3513037"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hashemi, N., Tahir, A., and Rasheed, S. (2022). An empirical study of flaky tests in javascript. Proceedings of the 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE.","DOI":"10.1109\/ICSME55016.2022.00011"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Parry, O., Kapfhammer, G.M., Hilton, M., and McMinn, P. (2022). Evaluating features for machine learning detection of order-and non-order-dependent flaky tests. Proceedings of the 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), IEEE Computer Society.","DOI":"10.1109\/ICST53961.2022.00021"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Alshammari, A., Morris, C., Hilton, M., and Bell, J. (2021). Flakeflagger: Predicting flakiness without rerunning tests. Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE), IEEE.","DOI":"10.1109\/ICSE43902.2021.00140"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1912","DOI":"10.1109\/TSE.2022.3201209","article-title":"Flakify: A black-box, language model-based predictor for flaky tests","volume":"49","author":"Fatima","year":"2022","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Qin, Y., Wang, S., Liu, K., Lin, B., Wu, H., Li, L., Mao, X., and Bissyand\u00e9, T.F. (2022). Peeler: Learning to effectively predict flakiness without running tests. Proceedings of the 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE.","DOI":"10.1109\/ICSME55016.2022.00031"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1007\/s10664-022-10227-1","article-title":"Static test flakiness prediction: How far can we go?","volume":"27","author":"Pontillo","year":"2022","journal-title":"Empir. Softw. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1109\/TSE.2019.2904957","article-title":"Today was a good day: The daily life of software developers","volume":"47","author":"Meyer","year":"2019","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Eck, M., Palomba, F., Castelluccio, M., and Bacchelli, A. (2019, January 26\u201330). Understanding flaky tests: The developer\u2019s perspective. Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia.","DOI":"10.1145\/3338906.3338945"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1145\/3672089.3672100","article-title":"Summary of the 1st International Flaky Test Workshop (FTW 2024)","volume":"49","author":"Parry","year":"2024","journal-title":"ACM SIGSOFT Softw. Eng. Notes"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ziftci, C., and Cavalcanti, D. (2020). De-flake your tests: Automatically locating root causes of flaky tests in code at google. Proceedings of the 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE.","DOI":"10.1109\/ICSME46990.2020.00083"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"107394","DOI":"10.1016\/j.infsof.2023.107394","article-title":"Test code flakiness in mobile apps: The developer\u2019s perspective","volume":"168","author":"Pontillo","year":"2024","journal-title":"Inf. Softw. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alshammari, A., Ammann, P., Hilton, M., and Bell, J. (2024). 230,439 Test Failures Later: An Empirical Evaluation of Flaky Failure Classifiers. Proceedings of the 2024 IEEE Conference on Software Testing, Verification and Validation (ICST), IEEE.","DOI":"10.1109\/ICST60714.2024.00031"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Alshammari, A., Ammann, P., Hilton, M., and Bell, J. (2024). A Study of Flaky Failure De-Duplication to Identify Unreliably Killed Mutants. Proceedings of the 2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), IEEE.","DOI":"10.1109\/ICSTW60967.2024.00054"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cordy, M., Rwemalika, R., Franci, A., Papadakis, M., and Harman, M. (2022, January 21\u201329). Flakime: Laboratory-controlled test flakiness impact assessment. Proceedings of the 44th International Conference on Software Engineering, Pittsburgh, PA, USA.","DOI":"10.1145\/3510003.3510194"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3104","DOI":"10.1109\/TSE.2024.3462251","article-title":"The effects of computational resources on flaky tests","volume":"50","author":"Silva","year":"2024","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_33","unstructured":"Bell, J., Legunsen, O., Hilton, M., Eloussi, L., Yung, T., and Marinov, D. (June, January 27). DeFlaker: Automatically detecting flaky tests. Proceedings of the 40th International Conference on Software Engineering, Gethenburg, Sweden."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, C., and Shi, A. (2022). Evolution-aware detection of order-dependent flaky tests. Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery.","DOI":"10.1145\/3533767.3534404"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, C., Khosravi, M.M., Lam, W., and Shi, A. (2023). Systematically producing test orders to detect order-dependent flaky tests. Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery.","DOI":"10.1145\/3597926.3598083"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wei, A., Yi, P., Li, Z., Xie, T., Marinov, D., and Lam, W. (2022). Preempting flaky tests via non-idempotent-outcome tests. Proceedings of the 44th International Conference on Software Engineering, Association for Computing Machinery.","DOI":"10.1145\/3510003.3510170"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wang, R., Chen, Y., and Lam, W. (2022). iPFlakies: A framework for detecting and fixing python order-dependent flaky tests. Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings, Association for Computing Machinery.","DOI":"10.1145\/3510454.3516846"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Li, C., Zhu, C., Wang, W., and Shi, A. (2022). Repairing order-dependent flaky tests via test generation. Proceedings of the 44th International Conference on Software Engineering, Association for Computing Machinery.","DOI":"10.1145\/3510003.3510173"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Rahman, S., and Shi, A. (2024). FlakeSync: Automatically repairing async flaky tests. Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering, Association for Computing Machinery.","DOI":"10.1145\/3597503.3639115"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chen, Y. (2024). Flakiness repair in the era of large language models. Proceedings of the 2024 IEEE\/ACM 46th International Conference on Software Engineering: Companion Proceedings, Association for Computing Machinery.","DOI":"10.1145\/3639478.3641227"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chen, Y., and Jabbarvand, R. (2024). Neurosymbolic repair of test flakiness. Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery.","DOI":"10.1145\/3650212.3680369"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Leesatapornwongsa, T., Ren, X., and Nath, S. (2022). FlakeRepro: Automated and efficient reproduction of concurrency-related flaky tests. Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Association for Computing Machinery.","DOI":"10.1145\/3540250.3558956"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1109\/QCE60285.2024.10274","article-title":"Automated flakiness detection in quantum software bug reports","volume":"Volume 2","author":"Zhang","year":"2024","journal-title":"Proceedings of the 2024 IEEE International Conference on Quantum Computing and Engineering (QCE)"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1007\/s10664-023-10433-5","article-title":"Evaluating the impact of flaky simulators on testing autonomous driving systems","volume":"29","author":"Amini","year":"2024","journal-title":"Empir. Softw. Eng."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Gruber, M., Roslan, M.F., Parry, O., Scharnb\u00f6ck, F., McMinn, P., and Fraser, G. (2024). Do automatic test generation tools generate flaky tests?. Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering, Association for Computing Machinery.","DOI":"10.1145\/3597503.3608138"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., and Jiang, D. (2020). Codebert: A pre-trained model for programming and natural languages. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lam, W., Oei, R., Shi, A., Marinov, D., and Xie, T. (2019). iDFlakies: A framework for detecting and partially classifying flaky tests. Proceedings of the 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), IEEE.","DOI":"10.1109\/ICST.2019.00038"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wei, A., Yi, P., Xie, T., Marinov, D., and Lam, W. (2021). Probabilistic and systematic coverage of consecutive test-method pairs for detecting order-dependent flaky tests. Tools and Algorithms for the Construction and Analysis of Systems: 27th International Conference, TACAS 2021, Springer.","DOI":"10.26226\/morressier.604907f41a80aac83ca25cf5"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/15\/6\/372\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T04:45:51Z","timestamp":1781153151000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/15\/6\/372"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,7]]},"references-count":48,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["computers15060372"],"URL":"https:\/\/doi.org\/10.3390\/computers15060372","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,7]]}}}