{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T01:54:11Z","timestamp":1776909251935,"version":"3.51.2"},"reference-count":81,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T00:00:00Z","timestamp":1718841600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T00:00:00Z","timestamp":1718841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001804","name":"Canada Research Chairs","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100018693","name":"HORIZON EUROPE Framework Programme","doi-asserted-by":"publisher","award":["No. 957254"],"award-info":[{"award-number":["No. 957254"]}],"id":[{"id":"10.13039\/100018693","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With the increasing complexity and scope of software systems, their dependability is crucial. The analysis of log data recorded during system execution can enable engineers to automatically predict failures at run time. Several Machine Learning (ML) techniques, including traditional ML and Deep Learning (DL), have been proposed to automate such tasks. However, current empirical studies are limited in terms of covering all main DL types\u2014Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and transformer\u2014as well as examining them on a wide range of diverse datasets. In this paper, we aim to address these issues by systematically investigating the combination of log data embedding strategies and DL types for failure prediction. To that end, we propose a modular architecture to accommodate various configurations of embedding strategies and DL-based encoders. To further investigate how dataset characteristics such as dataset size and failure percentage affect model accuracy, we synthesised 360 datasets, with varying characteristics, for three distinct system behavioural models, based on a systematic and automated generation approach. Using the F1 score metric, our results show that the best overall performing configuration is a CNN-based encoder with Logkey2vec. Additionally, we provide specific dataset conditions, namely a dataset size <jats:inline-formula><jats:alternatives><jats:tex-math>$$&gt;350$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mo>&gt;<\/mml:mo>\n                    <mml:mn>350<\/mml:mn>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> or a failure percentage <jats:inline-formula><jats:alternatives><jats:tex-math>$$&gt;7.5\\%$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mo>&gt;<\/mml:mo>\n                    <mml:mn>7.5<\/mml:mn>\n                    <mml:mo>%<\/mml:mo>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, under which this configuration demonstrates high accuracy for failure prediction.<\/jats:p>","DOI":"10.1007\/s10664-024-10501-4","type":"journal-article","created":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T13:02:12Z","timestamp":1718888532000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Systematic Evaluation of Deep Learning Models for Log-based Failure Prediction"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-8755-3323","authenticated-orcid":false,"given":"Fatemeh","family":"Hadadi","sequence":"first","affiliation":[]},{"given":"Joshua H.","family":"Dawes","sequence":"additional","affiliation":[]},{"given":"Donghwan","family":"Shin","sequence":"additional","affiliation":[]},{"given":"Domenico","family":"Bianculli","sequence":"additional","affiliation":[]},{"given":"Lionel","family":"Briand","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,6,20]]},"reference":[{"key":"10501_CR1","doi-asserted-by":"publisher","unstructured":"Basin D, Dardinier T, Heimes L, Krsti\u0107 S, Raszyk M, Schneider J, Traytel D (2020) A formally verified, optimized monitor for metric first-order dynamic logic. In: Automated Reasoning: 10th International joint conference, IJCAR 2020, Paris, France, July 1\u20134, 2020, Proceedings, Part I, Springer-Verlag, Berlin, Heidelberg, pp 432\u2013453, https:\/\/doi.org\/10.1007\/978-3-030-51074-9_25","DOI":"10.1007\/978-3-030-51074-9_25"},{"key":"10501_CR2","doi-asserted-by":"publisher","DOI":"10.1002\/9781118393994","volume-title":"Reliability and availability of cloud computing","author":"E Bauer","year":"2012","unstructured":"Bauer E, Adams R (2012) Reliability and availability of cloud computing. John Wiley & Sons"},{"key":"10501_CR3","unstructured":"Black PE (2020) Strongly connected component. Dictionary of Algorithms and Data Structures https:\/\/www.nist.gov\/dads\/HTML\/stronglyConnectedCompo.html"},{"key":"10501_CR4","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/978-3-540-31848-4_9","volume":"3395","author":"J Blom","year":"2005","unstructured":"Blom J, Hessel A, Jonsson B, Pettersson P (2005) Specifying and generating test cases using observer automata. Lecture Notes Comput Sci 3395:125\u2013139. https:\/\/doi.org\/10.1007\/978-3-540-31848-4_9","journal-title":"Lecture Notes Comput Sci"},{"key":"10501_CR5","doi-asserted-by":"crossref","unstructured":"Bogatinovski J, Nedelkoski S, Wu L, Cardoso J, Kao O (2022) Failure identification from unstable log data using deep learning. In: 2022 22nd IEEE International symposium on cluster, cloud and internet computing (CCGrid) pp 346\u2013355, https:\/\/api.semanticscholar.org\/CorpusID:247996709","DOI":"10.1109\/CCGrid54584.2022.00044"},{"key":"10501_CR6","doi-asserted-by":"publisher","unstructured":"Bombarda A, Gargantini A (2020) An Automata-Based Generation Method for Combinatorial Sequence Testing of Finite State Machines. In: Proceedings - 2020 IEEE 13th International conference on software testing, verification and validation workshops, ICSTW 2020 pp 157\u2013166. https:\/\/doi.org\/10.1109\/ICSTW50294.2020.00036","DOI":"10.1109\/ICSTW50294.2020.00036"},{"issue":"1","key":"10501_CR7","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45(1):5\u201332. https:\/\/doi.org\/10.1023\/A:1010933404324","journal-title":"Mach Learn"},{"key":"10501_CR8","volume-title":"Classification and Regression Trees","author":"L Breiman","year":"1984","unstructured":"Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and Regression Trees. Wadsworth"},{"key":"10501_CR9","unstructured":"Breiman L, Friedman JH, Olshen RA, Stone CJ (1984b) Classification and Regression Trees. Chapman and Hall\/CRC"},{"key":"10501_CR10","doi-asserted-by":"publisher","unstructured":"Carvalho TP, Soares FAAMN, Vita R, da P Francisco R, Basto JP, Alcal\u00e1 SGS, (2019) A systematic literature review of machine learning methods applied to predictive maintenance. Comput & Industrial Eng 137:106024. https:\/\/doi.org\/10.1016\/j.cie.2019.106024, https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0360835219304838","DOI":"10.1016\/j.cie.2019.106024"},{"key":"10501_CR11","doi-asserted-by":"publisher","unstructured":"Chen Y, Yang X, Lin Q, Zhang D, Dong H, Xu Y, Li H, Kang Y, Zhang H, Gao F, Xu Z, Dang Y (2019) Outage prediction and diagnosis for cloud service systems. In: The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019 pp 2659\u20132665. https:\/\/doi.org\/10.1145\/3308558.3313501","DOI":"10.1145\/3308558.3313501"},{"key":"10501_CR12","unstructured":"Chen Y, Li L, Li W, Guo Q, Du Z, Xu Z (2022) AI Computing Systems: An Application Driven Perspective. Elsevier Sci https:\/\/books.google.ca\/books?id=RSWJEAAAQBAJ"},{"key":"10501_CR13","doi-asserted-by":"publisher","unstructured":"Chen Z, Liu J, Gu W, Su Y, Lyu MR (2021) Experience report: Deep learning-based system log analysis for anomaly detection. https:\/\/doi.org\/10.48550\/ARXIV.2107.05908, https:\/\/arxiv.org\/abs\/2107.05908","DOI":"10.48550\/ARXIV.2107.05908"},{"key":"10501_CR14","doi-asserted-by":"crossref","unstructured":"Cho K, Van\u00a0Merri\u00ebnboer B, Bahdanau D, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078","DOI":"10.3115\/v1\/D14-1179"},{"key":"10501_CR15","doi-asserted-by":"crossref","unstructured":"Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: IEEE Conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2017.195"},{"issue":"3","key":"10501_CR16","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/BF00994018","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273\u2013297","journal-title":"Mach Learn"},{"key":"10501_CR17","doi-asserted-by":"publisher","unstructured":"Cotroneo D, De Simone L, Liguori P, Natella R, Bidokhti N (2019) How bad can a bug get? An empirical analysis of software failures in the OpenStack cloud computing platform. In: ESEC\/FSE 2019 - Proceedings of the 2019 27th ACM Joint Meeting European software engineering conference and symposium on the foundations of software engineering pp 200\u2013211, https:\/\/doi.org\/10.1145\/3338906.3338916, arXiv:1907.04055","DOI":"10.1145\/3338906.3338916"},{"key":"10501_CR18","doi-asserted-by":"publisher","unstructured":"Das A, Mueller F, Siegel C, Vishnu A (2018) Desh: Deep learning for system health prediction of lead times to failure in HPC. In: HPDC 2018 - Proceedings of the 2018 International symposium on high-performance parallel and distributed computing pp 40\u201351, https:\/\/doi.org\/10.1145\/3208040.3208051","DOI":"10.1145\/3208040.3208051"},{"key":"10501_CR19","doi-asserted-by":"publisher","unstructured":"Das A, Mueller F, Rountree B (2020) Aarohi: Making Real-Time Node Failure Prediction Feasible. In: Proceedings - 2020 IEEE 34th International parallel and distributed processing symposium, IPDPS 2020 pp 1092\u20131101, https:\/\/doi.org\/10.1109\/IPDPS47924.2020.00115","DOI":"10.1109\/IPDPS47924.2020.00115"},{"issue":"6","key":"10501_CR20","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1002\/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9","volume":"41","author":"S Deerwester","year":"1990","unstructured":"Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391\u2013407","journal-title":"J Am Soc Inf Sci"},{"key":"10501_CR21","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805"},{"key":"10501_CR22","unstructured":"Digital Research Alliance of Canada (2016) https:\/\/alliancecan.ca\/, accessed: March 2, 2023"},{"key":"10501_CR23","unstructured":"Ding M, Zhou C, Yang H, Tang J (2020) Cogltx: Applying bert to long texts. In: Neural information processing systems"},{"key":"10501_CR24","doi-asserted-by":"publisher","unstructured":"Du M, Li F, Zheng G, Srikumar V (2017) Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on computer and communications security, association for computing machinery, New York, NY, USA, CCS \u201917, p 1285\u20131298, https:\/\/doi.org\/10.1145\/3133956.3134015,","DOI":"10.1145\/3133956.3134015"},{"key":"10501_CR25","unstructured":"Dwivedi VP, Luu AT, Laurent T, Bengio Y, Bresson X (2021) Graph neural networks with learnable structural and positional representations. arXiv:2110.07875"},{"key":"10501_CR26","first-page":"3133","volume":"15","author":"M Fern\u00e1ndez-Delgado","year":"2014","unstructured":"Fern\u00e1ndez-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15:3133\u20133181","journal-title":"J Mach Learn Res"},{"key":"10501_CR27","unstructured":"Foundation CC (2023) Common crawl corpus. https:\/\/commoncrawl.org\/"},{"issue":"10","key":"10501_CR28","doi-asserted-by":"publisher","first-page":"2451","DOI":"10.1162\/089976600300015015","volume":"12","author":"FA Gers","year":"2000","unstructured":"Gers FA, Schmidhuber JA, Cummins FA (2000) Learning to forget: Continual prediction with lstm. Neural Comput 12(10):2451\u20132471. https:\/\/doi.org\/10.1162\/089976600300015015","journal-title":"Neural Comput"},{"key":"10501_CR29","doi-asserted-by":"publisher","unstructured":"Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J, Chen T (2018) Recent advances in convolutional neural networks. Pattern Recognition 77:354\u2013377, https:\/\/doi.org\/10.1016\/j.patcog.2017.10.013, https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0031320317304120","DOI":"10.1016\/j.patcog.2017.10.013"},{"key":"10501_CR30","doi-asserted-by":"publisher","unstructured":"Guo H, Yuan S, Wu X (2021) Logbert: Log anomaly detection via bert. In: 2021 International joint conference on neural networks (IJCNN), pp 1\u20138, https:\/\/doi.org\/10.1109\/IJCNN52387.2021.9534113","DOI":"10.1109\/IJCNN52387.2021.9534113"},{"key":"10501_CR31","doi-asserted-by":"publisher","unstructured":"Hadadi F, Dawes J, Shin D, Bianculli D, Briand L (2024) Replication package. https:\/\/doi.org\/10.6084\/m9.figshare.22219111, https:\/\/figshare.com\/articles\/software\/Replication_Package\/22219111","DOI":"10.6084\/m9.figshare.22219111"},{"key":"10501_CR32","doi-asserted-by":"crossref","unstructured":"He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21:1263\u20131284, https:\/\/api.semanticscholar.org\/CorpusID:206742563","DOI":"10.1109\/TKDE.2008.239"},{"key":"10501_CR33","doi-asserted-by":"publisher","unstructured":"He S, He P, Chen Z, Yang T, Su Y, Lyu MR (2021) A Survey on Automated Log Analysis for Reliability Engineering. ACM Comput Surv 54(6), https:\/\/doi.org\/10.1145\/3460345, arXiv:2009.07237","DOI":"10.1145\/3460345"},{"issue":"8","key":"10501_CR34","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780. https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735","journal-title":"Neural Comput"},{"issue":"8","key":"10501_CR35","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780","journal-title":"Neural Comput"},{"issue":"4","key":"10501_CR36","doi-asserted-by":"publisher","first-page":"2064","DOI":"10.1109\/TNSM.2020.3034647","volume":"17","author":"S Huang","year":"2020","unstructured":"Huang S, Liu Y, Fung C, He R, Zhao Y, Yang H, Luan Z (2020) HitAnomaly: Hierarchical Transformers for Anomaly Detection in System Log. IEEE Trans Netw Service Manag 17(4):2064\u20132076. https:\/\/doi.org\/10.1109\/TNSM.2020.3034647","journal-title":"IEEE Trans Netw Service Manag"},{"key":"10501_CR37","doi-asserted-by":"publisher","unstructured":"Huang Z, Xu W, Yu K (2015) Bidirectional lstm-crf models for sequence tagging. https:\/\/doi.org\/10.48550\/ARXIV.1508.01991, arXiv:1508.01991","DOI":"10.48550\/ARXIV.1508.01991"},{"key":"10501_CR38","doi-asserted-by":"publisher","unstructured":"Johnson JM, Khoshgoftaar TM (2019) Survey on deep learning with class imbalance. J Big Data 6(1), https:\/\/doi.org\/10.1186\/s40537-019-0192-5","DOI":"10.1186\/s40537-019-0192-5"},{"key":"10501_CR39","unstructured":"Joulin A, Grave E, Bojanowski P, Douze M, J\u00e9gou H, Mikolov T (2016) Fasttext.zip: Compressing text classification models. arXiv:1612.03651"},{"key":"10501_CR40","doi-asserted-by":"crossref","unstructured":"Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:1408.5882","DOI":"10.3115\/v1\/D14-1181"},{"key":"10501_CR41","unstructured":"Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, arXiv:1412.6980"},{"key":"10501_CR42","unstructured":"Kluge F, Rochange C, Ungerer T (2017) EMSBench: Benchmark and Testbed for Reactive Real-Time Systems. Leibniz Trans Embedded Syst 4(2):02\u20131\u201302:23, https:\/\/ojs.dagstuhl.de\/index.php\/lites\/article\/view\/LITES-v004-i002-a002"},{"key":"10501_CR43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-60508-7_27","volume-title":"A Benchmark Generator for Online First-Order Monitoring","author":"S Krsti\u0107","year":"2020","unstructured":"Krsti\u0107 S, Schneider J (2020) A Benchmark Generator for Online First-Order Monitoring, vol 12399. Springer International Publishing, LNCS. https:\/\/doi.org\/10.1007\/978-3-030-60508-7_27"},{"key":"10501_CR44","doi-asserted-by":"publisher","unstructured":"Le VH, Zhang H (2021) Log-based anomaly detection without log parsing. In: 2021 36th IEEE\/ACM International conference on automated software engineering (ASE), pp 492\u2013504, https:\/\/doi.org\/10.1109\/ASE51524.2021.9678773","DOI":"10.1109\/ASE51524.2021.9678773"},{"key":"10501_CR45","doi-asserted-by":"publisher","unstructured":"Le VH, Zhang H (2022) Log-based anomaly detection with deep learning: How far are we? In: Proceedings of the 44th international conference on software engineering, association for computing machinery, New York, NY, USA, ICSE \u201922, p 1356\u20131367, https:\/\/doi.org\/10.1145\/3510003.3510155","DOI":"10.1145\/3510003.3510155"},{"key":"10501_CR46","doi-asserted-by":"publisher","unstructured":"Li X, Chen P, Jing L, He Z, Yu G (2020) Swisslog: Robust and unified deep learning based log anomaly detection for diverse faults. In: 2020 IEEE 31st International symposium on software reliability engineering (ISSRE), IEEE computer society, Los Alamitos, CA, USA, pp 92\u2013103, https:\/\/doi.org\/10.1109\/ISSRE5003.2020.00018, https:\/\/doi.ieeecomputersociety.org\/10.1109\/ISSRE5003.2020.00018","DOI":"10.1109\/ISSRE5003.2020.00018"},{"key":"10501_CR47","doi-asserted-by":"publisher","unstructured":"Lin Q, Hsieh K, Dang Y, Zhang H, Sui K, Xu Y, Lou JG, Li C, Wu Y, Yao R, Chintalapati M, Zhang D (2018) Predicting node failure in cloud service systems. In: Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, association for computing machinery, New York, NY, USA, ESEC\/FSE 2018, p 480\u2013490, https:\/\/doi.org\/10.1145\/3236024.3236060","DOI":"10.1145\/3236024.3236060"},{"key":"10501_CR48","unstructured":"Lipton ZC (2015) A critical review of recurrent neural networks for sequence learning. arXiv:1506.00019"},{"key":"10501_CR49","doi-asserted-by":"publisher","unstructured":"Liu X, He Y, Liu H, Zhang J, Liu B, Peng X, Xu J, Zhang J, Zhou A, Sun P, Zhu K, Nishi A, Zhu D, Zhang K (2020) Smart Server Crash Prediction in Cloud Service Data Center. In: 2020 19th IEEE Intersociety conference on thermal and thermomechanical phenomena in electronic systems (ITherm), https:\/\/doi.org\/10.1109\/ITherm45881.2020.9190321","DOI":"10.1109\/ITherm45881.2020.9190321"},{"key":"10501_CR50","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: A robustly optimized bert pretraining approach. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP)"},{"key":"10501_CR51","doi-asserted-by":"publisher","first-page":"21929","DOI":"10.1109\/ACCESS.2018.2811530","volume":"6","author":"S Lu","year":"2018","unstructured":"Lu S, Wei X, Li Y, Wang L (2018) Detecting anomaly in big data system logs using convolutional neural network. IEEE Access 6:21929\u201321940. https:\/\/doi.org\/10.1109\/ACCESS.2018.2811530","journal-title":"IEEE Access"},{"issue":"1","key":"10501_CR52","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1145\/272991.272995","volume":"8","author":"M Matsumoto","year":"1998","unstructured":"Matsumoto M, Nishimura T (1998) Mersenne Twister: A 623-Dimensionally Equidistributed Uniform Pseudo-Random Number Generator. ACM Trans Model Comput Simulation 8(1):3\u201330. https:\/\/doi.org\/10.1145\/272991.272995","journal-title":"ACM Trans Model Comput Simulation"},{"key":"10501_CR53","doi-asserted-by":"crossref","unstructured":"Meng W, Liu Y, Zhu Y, Zhang S, Pei D, Liu Y, Chen Y, Zhang R, Tao S, Sun P, Zhou R (2019) Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In: International joint conference on artificial intelligence","DOI":"10.24963\/ijcai.2019\/658"},{"key":"10501_CR54","doi-asserted-by":"publisher","unstructured":"Meng W, Liu Y, Huang Y, Zhang S, Zaiter F, Chen B, Pei D (2020) A semantic-aware representation framework for online log analysis. In: 2020 29th International conference on computer communications and networks (ICCCN), pp 1\u20137, https:\/\/doi.org\/10.1109\/ICCCN49398.2020.9209707","DOI":"10.1109\/ICCCN49398.2020.9209707"},{"key":"10501_CR55","unstructured":"Mikolov T, Chen K, Corrado GS, Dean J (2013) Efficient estimation of word representations in vector space. In: International conference on learning representations"},{"key":"10501_CR56","doi-asserted-by":"publisher","unstructured":"Nedelkoski S, Bogatinovski J, Acker A, Cardoso J, Kao O (2020) Self-attentive classification-based anomaly detection in unstructured logs. Proceedings - IEEE international conference on data mining, ICDM 2020-Novem(Icdm):1196\u20131201, https:\/\/doi.org\/10.1109\/ICDM50108.2020.00148, arXiv:2008.09340","DOI":"10.1109\/ICDM50108.2020.00148"},{"key":"10501_CR57","doi-asserted-by":"publisher","unstructured":"Notaro P, Cardoso J, Gerndt M (2021) A survey of aiops methods for failure management. ACM Trans Intell Syst Technol 12(6), https:\/\/doi.org\/10.1145\/3483424","DOI":"10.1145\/3483424"},{"key":"10501_CR58","doi-asserted-by":"publisher","unstructured":"O\u2019Shea K, Nash R (2015) An introduction to convolutional neural networks. https:\/\/doi.org\/10.48550\/ARXIV.1511.08458, arXiv:1511.08458","DOI":"10.48550\/ARXIV.1511.08458"},{"key":"10501_CR59","unstructured":"Package RP (2019) https:\/\/docs.python.org\/3\/library\/random.html, accessed 2022-11-14"},{"key":"10501_CR60","doi-asserted-by":"crossref","unstructured":"Prechelt L (1998) Early stopping-but when? In: Neural Networks: Tricks of the Trade, Springer, pp 55\u201369","DOI":"10.1007\/3-540-49430-8_3"},{"key":"10501_CR61","unstructured":"Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. In: Neural information processing systems"},{"key":"10501_CR62","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139058452","author":"A Rajaraman","year":"2014","unstructured":"Rajaraman A, Leskovec J, Ullman J (2014) Mining of Massive Datasets. Cambridge University Press. https:\/\/doi.org\/10.1017\/CBO9781139058452","journal-title":"Cambridge University Press"},{"issue":"4","key":"10501_CR63","doi-asserted-by":"publisher","first-page":"879","DOI":"10.1007\/s10664-014-9303-2","volume":"20","author":"B Russo","year":"2015","unstructured":"Russo B, Succi G, Pedrycz W (2015) Mining system logs to learn error predictors: a case study of a telemetry system. Empirical Softw Eng 20(4):879\u2013927. https:\/\/doi.org\/10.1007\/s10664-014-9303-2","journal-title":"Empirical Softw Eng"},{"key":"10501_CR64","doi-asserted-by":"publisher","unstructured":"Sahoo RK, Oliner AJ, Rish I, Gupta M, Moreira JE, Ma S, Vilalta R, Sivasubramaniam A (2003) Critical event prediction for proactive management in large-scale computer clusters. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining pp 426\u2013435, https:\/\/doi.org\/10.1145\/956750.956799","DOI":"10.1145\/956750.956799"},{"key":"10501_CR65","doi-asserted-by":"publisher","unstructured":"Salfner F, Lenk M, Malek M (2010) A survey of online failure prediction methods. ACM Comput Surveys 42(3), https:\/\/doi.org\/10.1145\/1670679.1670680","DOI":"10.1145\/1670679.1670680"},{"issue":"11","key":"10501_CR66","doi-asserted-by":"publisher","first-page":"2673","DOI":"10.1109\/78.650093","volume":"45","author":"M Schuster","year":"1997","unstructured":"Schuster M, Paliwal K (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673\u20132681. https:\/\/doi.org\/10.1109\/78.650093","journal-title":"IEEE Trans Signal Process"},{"key":"10501_CR67","doi-asserted-by":"publisher","unstructured":"Shin D, Bianculli D, Briand L (2022) Prins: Scalable model inference for component-based system logs. Empirical Softw Engg 27(4), https:\/\/doi.org\/10.1007\/s10664-021-10111-4","DOI":"10.1007\/s10664-021-10111-4"},{"key":"10501_CR68","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1007\/978-3-030-32381-3_16","volume-title":"Chinese Computational Linguistics","author":"C Sun","year":"2019","unstructured":"Sun C, Qiu X, Xu Y, Huang X (2019) How to fine-tune bert for text classification? In: Sun M, Huang X, Ji H, Liu Z, Liu Y (eds) Chinese Computational Linguistics. Springer International Publishing, Cham, pp 194\u2013206"},{"key":"10501_CR69","unstructured":"Tauber A (2018) exrex: Irregular methods for regular expressions. https:\/\/github.com\/asciimoo\/exrex, accessed 2022-11-14"},{"key":"10501_CR70","doi-asserted-by":"crossref","unstructured":"Upton G, Cook I (2008) A Dictionary of Statistics. Oxford Paperback Reference, OUP Oxford, https:\/\/books.google.ca\/books?id=u97pzxRjaCQC","DOI":"10.1093\/acref\/9780199541454.001.0001"},{"key":"10501_CR71","unstructured":"Vaswani A, Shazeer NM, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. arXiv:1706.03762"},{"key":"10501_CR72","doi-asserted-by":"publisher","unstructured":"Walkinshaw N, Taylor R, Derrick J (2013) Inferring extended finite state machine models from software executions. In: 2013 20th Working conference on reverse engineering (WCRE), pp 301\u2013310, https:\/\/doi.org\/10.1109\/WCRE.2013.6671305","DOI":"10.1109\/WCRE.2013.6671305"},{"key":"10501_CR73","doi-asserted-by":"publisher","unstructured":"Weijie D, Yunyi L, Jing Z, Xuchen S (2021) Long text classification based on bert. In: 2021 IEEE 5th Information Technology,Networking,Electronic and Automation Control Conference (ITNEC), vol\u00a05, pp 1147\u20131151, https:\/\/doi.org\/10.1109\/ITNEC52019.2021.9587007","DOI":"10.1109\/ITNEC52019.2021.9587007"},{"key":"10501_CR74","doi-asserted-by":"crossref","unstructured":"Wu X, Li H, Khomh F (2023) On the effectiveness of log representation for log-based anomaly detection. arXiv:2308.08736","DOI":"10.1007\/s10664-023-10364-1"},{"key":"10501_CR75","unstructured":"Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K, Klingner J, Shah A, Johnson M, Liu X, Kaiser L, Gouws S, Kato Y, Kudo T, Kazawa H, Stevens K, Kurian G, Patil N, Wang W, Young C, Smith J, Riesa J, Rudnick A, Vinyals O, Corrado G, Hughes M, Dean J (2016) Google\u2019s neural machine translation system: Bridging the gap between human and machine translation. arXiv:1609.08144"},{"key":"10501_CR76","doi-asserted-by":"publisher","unstructured":"Xie Y, Zhang H, Babar MA (2022) Loggd: Detecting anomalies from system logs with graph neural networks. In: 2022 IEEE 22nd International conference on software quality, reliability and security (QRS), pp 299\u2013310, https:\/\/doi.org\/10.1109\/QRS57517.2022.00039","DOI":"10.1109\/QRS57517.2022.00039"},{"key":"10501_CR77","doi-asserted-by":"crossref","unstructured":"Xu P, Kumar D, Yang W, Zi W, Tang K, Huang C, Cheung JCK, Prince S, Cao Y (2020) Optimizing deeper transformers on small datasets. In: Annual meeting of the association for computational linguistics","DOI":"10.18653\/v1\/2021.acl-long.163"},{"key":"10501_CR78","doi-asserted-by":"publisher","unstructured":"Yamanishi K, Maruyama Y (2005) Dynamic syslog mining for network failure monitoring. In: Proceedings of the Eleventh ACM SIGKDD International conference on knowledge discovery in data mining, association for computing machinery, New York, NY, USA, KDD \u201905, p 499\u2013508, https:\/\/doi.org\/10.1145\/1081870.1081927","DOI":"10.1145\/1081870.1081927"},{"key":"10501_CR79","doi-asserted-by":"publisher","unstructured":"Yang L, Chen J, Wang Z, Wang W, Jiang J, Dong X, Zhang W (2021) Semi-supervised log-based anomaly detection via probabilistic label estimation. In: 2021 IEEE\/ACM 43rd International conference on software engineering (ICSE), pp 1448\u20131460, https:\/\/doi.org\/10.1109\/ICSE43902.2021.00130","DOI":"10.1109\/ICSE43902.2021.00130"},{"key":"10501_CR80","doi-asserted-by":"publisher","unstructured":"Zhang S, Liu Y, Meng W, Luo Z, Bu J, Yang S, Liang P, Pei D, Xu J, Zhang Y, Chen Y, Dong H, Qu X, Song L (2018) Prefix: Switch failure prediction in datacenter networks. Proc ACM Meas Anal Comput Syst 2(1):2:1\u20132:29, https:\/\/doi.org\/10.1145\/3179405","DOI":"10.1145\/3179405"},{"key":"10501_CR81","doi-asserted-by":"publisher","unstructured":"Zhang X, Xu Y, Lin Q, Qiao B, Zhang H, Dang Y, Xie C, Yang X, Cheng Q, Li Z, Chen J, He X, Yao R, Lou JG, Chintalapati M, Shen F, Zhang D (2019) Robust log-based anomaly detection on unstable log data. In: ESEC\/FSE 2019 - Proceedings of the 2019 27th ACM joint meeting european software engineering conference and symposium on the foundations of software engineering pp 807\u2013817, https:\/\/doi.org\/10.1145\/3338906.3338931","DOI":"10.1145\/3338906.3338931"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10501-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-024-10501-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10501-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T03:45:02Z","timestamp":1725594302000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-024-10501-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,20]]},"references-count":81,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["10501"],"URL":"https:\/\/doi.org\/10.1007\/s10664-024-10501-4","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,20]]},"assertion":[{"value":"17 May 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 June 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest"}}],"article-number":"105"}}