{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T16:10:29Z","timestamp":1780675829843,"version":"3.54.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,4,16]],"date-time":"2024-04-16T00:00:00Z","timestamp":1713225600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,16]],"date-time":"2024-04-16T00:00:00Z","timestamp":1713225600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["298020 and 328058"],"award-info":[{"award-number":["298020 and 328058"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["298020 and 328058"],"award-info":[{"award-number":["298020 and 328058"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006196","name":"University of Oulu","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006196","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Autom Softw Eng"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With the growth of online services, IoT devices, and DevOps-oriented software development, software log anomaly detection is becoming increasingly important. Prior works mainly follow a traditional four-staged architecture (Preprocessor, Parser, Vectorizer, and Classifier). This paper proposes OneLog, which utilizes a single deep neural network instead of multiple separate components. OneLog harnesses convolutional neural network (CNN) at the character level to take digits, numbers, and punctuations, which were removed in prior works, into account alongside the main natural language text. We evaluate our approach in six message- and sequence-based data sets: HDFS, Hadoop, BGL, Thunderbird, Spirit, and Liberty. We experiment with Onelog with single-, multi-, and cross-project setups. Onelog offers state-of-the-art performance in our datasets. Onelog can utilize multi-project datasets simultaneously during training, which suggests our model can generalize between datasets. Multi-project training also improves Onelog performance making it ideal when limited training data is available for an individual project. We also found that cross-project anomaly detection is possible with a single project pair (Liberty and Spirit). Analysis of model internals shows that one log has multiple modes of detecting anomalies and that the model learns manually validated parsing rules for the log messages. We conclude that character-based CNNs are a promising approach toward end-to-end learning in log anomaly detection. They offer good performance and generalization over multiple datasets. We will make our scripts publicly available upon the acceptance of this paper.<\/jats:p>","DOI":"10.1007\/s10515-024-00428-x","type":"journal-article","created":{"date-parts":[[2024,4,16]],"date-time":"2024-04-16T18:02:53Z","timestamp":1713290573000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["OneLog: towards end-to-end software log anomaly detection"],"prefix":"10.1007","volume":"31","author":[{"given":"Shayan","family":"Hashemi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mika","family":"M\u00e4ntyl\u00e4","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,16]]},"reference":[{"issue":"7","key":"428_CR1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130140","volume":"10","author":"S Bach","year":"2015","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.-R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), e0130140 (2015)","journal-title":"PLoS ONE"},{"key":"428_CR2","unstructured":"Ballard, D.H.: Modular learning in neural networks. In: Aaai, vol. 647, pp. 279\u2013284 (1987)"},{"key":"428_CR3","unstructured":"Bojarski, M., Del\u00a0Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., et\u00a0al.: End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)"},{"key":"428_CR4","doi-asserted-by":"crossref","unstructured":"Bromley, J., Guyon, I., LeCun, Y., S\u00e4ckinger, E., Shah, R.: Signature verification using a \u201csiamese\u201d time delay neural network. In: Advances in Neural Information Processing Systems, pp. 737\u2013744 (1994)","DOI":"10.1142\/9789812797926_0003"},{"key":"428_CR5","unstructured":"Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)"},{"key":"428_CR6","doi-asserted-by":"publisher","unstructured":"Du, M.,\u00a0Li, F.: Spell: streaming parsing of system event logs. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 859\u2013864 (2016). https:\/\/doi.org\/10.1109\/ICDM.2016.0103","DOI":"10.1109\/ICDM.2016.0103"},{"key":"428_CR7","doi-asserted-by":"crossref","unstructured":"Du, M., Li, F., Zheng, G., Srikumar, V.: Deeplog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1285\u20131298 (2017)","DOI":"10.1145\/3133956.3134015"},{"key":"428_CR8","doi-asserted-by":"crossref","unstructured":"Farzad, A., Gulliver, T.A.: Log message anomaly detection and classification using auto-b\/lstm and auto-gru. arXiv preprint arXiv:1911.08744 (2019)","DOI":"10.31224\/osf.io\/d4e6a"},{"key":"428_CR9","unstructured":"Glasmachers, T.: Limits of end-to-end learning. In: Asian Conference on Machine Learning, pp. 17\u201332. PMLR (2017)"},{"key":"428_CR10","doi-asserted-by":"crossref","unstructured":"Guo, H., Yuan, S., Wu, X.: Logbert: log anomaly detection via bert. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1\u20138. IEEE (2021)","DOI":"10.1109\/IJCNN52387.2021.9534113"},{"issue":"2","key":"428_CR11","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1007\/s10515-022-00365-7","volume":"29","author":"S Hashemi","year":"2022","unstructured":"Hashemi, S., M\u00e4ntyl\u00e4, M.: Sialog: detecting anomalies in software execution logs using the siamese network. Autom. Softw. Eng. 29(2), 61 (2022). https:\/\/doi.org\/10.1007\/s10515-022-00365-7","journal-title":"Autom. Softw. Eng."},{"issue":"8","key":"428_CR12","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735\u20131780 (1997)","journal-title":"Neural Comput."},{"key":"428_CR13","doi-asserted-by":"crossref","unstructured":"Le, V.-H., Zhang, H.: Log-based anomaly detection without log parsing. CoRR, arXiv:2108.01955 (2021)","DOI":"10.1109\/ASE51524.2021.9678773"},{"key":"428_CR14","unstructured":"LeCun, Y., et\u00a0al.: Lenet-5, convolutional neural networks. http:\/\/yann.lecun.com\/exdb\/lenet, 20(5), 14 (2015)"},{"key":"428_CR15","doi-asserted-by":"crossref","unstructured":"Lin, Q.,\u00a0Zhang, H.,\u00a0Lou, J.,\u00a0Zhang, Y.,\u00a0Chen, X.: Log clustering based problem identification for online service systems. In: 2016 IEEE\/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pp. 102\u2013111 (2016)","DOI":"10.1145\/2889160.2889232"},{"key":"428_CR16","doi-asserted-by":"crossref","unstructured":"Lu, S., Wei, X., Li, Y., Wang, L.: Detecting anomaly in big data system logs using convolutional neural network. In: 2018 IEEE 16th International Conference on Dependable, Autonomic and Secure Computing, 16th International Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC\/PiCom\/DataCom\/CyberSciTech), pp. 151\u2013158. IEEE (2018)","DOI":"10.1109\/DASC\/PiCom\/DataCom\/CyberSciTec.2018.00037"},{"key":"428_CR17","doi-asserted-by":"crossref","unstructured":"McInnes, L., Healy, J., Melville, J.: Umap: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)","DOI":"10.21105\/joss.00861"},{"key":"428_CR18","doi-asserted-by":"crossref","unstructured":"Meng, W., Liu, Y., Zhu, Y., Zhang, S., Pei, D., Liu, Y., Chen, Y., Zhang, R., Tao, S., Sun, P., et\u00a0al.: Loganomaly: unsupervised detection of sequential and quantitative anomalies in unstructured logs. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, vol.\u00a07, pp. 4739\u20134745 (2019)","DOI":"10.24963\/ijcai.2019\/658"},{"key":"428_CR19","doi-asserted-by":"crossref","unstructured":"Nedelkoski, S., Bogatinovski, J., Acker, A., Cardoso, J., Kao, O.: Self-attentive classification-based anomaly detection in unstructured logs. arXiv preprint arXiv:2008.09340 (2020)","DOI":"10.1109\/ICDM50108.2020.00148"},{"key":"428_CR20","doi-asserted-by":"publisher","unstructured":"Oliner, A.,\u00a0Stearley, J.: What supercomputers say: a study of five system logs. In: 37th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201907), pp. 575\u2013584 (2007). https:\/\/doi.org\/10.1109\/DSN.2007.103","DOI":"10.1109\/DSN.2007.103"},{"key":"428_CR21","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011)","journal-title":"J. Mach. Learn. Res."},{"key":"428_CR22","doi-asserted-by":"crossref","unstructured":"Shahapure, K.R., Nicholas, C.: Cluster quality analysis using silhouette score. In: 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), pp. 747\u2013748. IEEE (2020)","DOI":"10.1109\/DSAA49011.2020.00096"},{"key":"428_CR23","unstructured":"Shen, S., Cheng, Y., He, Z., He, W., Wu, H., Sun, M., Liu, Y.: Minimum risk training for neural machine translation. arXiv preprint arXiv:1512.02433 (2015)"},{"key":"428_CR24","unstructured":"Shrikumar, A., Greenside, P., Shcherbina, A., Kundaje, A.: Not just a black box: learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713 (2016)"},{"key":"428_CR25","unstructured":"Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: International Conference on Machine Learning, pp. 3145\u20133153. PMLR (2017)"},{"issue":"7587","key":"428_CR26","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484\u2013489 (2016)","journal-title":"Nature"},{"key":"428_CR27","doi-asserted-by":"crossref","unstructured":"Starczewski, A., Krzy\u017cak, A.: Performance evaluation of the silhouette index. In: International Conference on Artificial Intelligence and Soft Computing, pp. 49\u201358. Springer (2015)","DOI":"10.1007\/978-3-319-19369-4_5"},{"key":"428_CR28","unstructured":"Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International Conference on Machine Learning, pp. 3319\u20133328. PMLR (2017)"},{"key":"428_CR29","first-page":"2025","volume":"3","author":"E Winter","year":"2002","unstructured":"Winter, E.: The Shapley value. Handb. Game Theory Econ. Appl. 3, 2025\u20132054 (2002)","journal-title":"Handb. Game Theory Econ. Appl."},{"key":"428_CR30","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29044-2","volume-title":"Experimentation in Software Engineering","author":"C Wohlin","year":"2012","unstructured":"Wohlin, C., Runeson, P., H\u00f6st, M., Ohlsson, M.C., Regnell, B., Wessl\u00e9n, A.: Experimentation in Software Engineering. Springer, Berlin (2012)"},{"key":"428_CR31","unstructured":"Wu, R., Guo, C., Hannun, A., van\u00a0der Maaten, L.: Fixes that fail: self-defeating improvements in machine-learning systems. In:\u00a0Ranzato, M.,\u00a0Beygelzimer, A.,\u00a0Dauphin, Y., Liang, P.S.,\u00a0Wortman Vaughan, J. (eds.) Advances in Neural Information Processing Systems, vol.\u00a034, pp. 11745\u201311756. Curran Associates, Inc., 2021. https:\/\/proceedings.neurips.cc\/paper\/2021\/file\/619427579e7b067421f6aa89d4a8990c-Paper.pdf"},{"key":"428_CR32","doi-asserted-by":"crossref","unstructured":"Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.I.: Detecting large-scale system problems by mining console logs. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 117\u2013132 (2009)","DOI":"10.1145\/1629575.1629587"},{"key":"428_CR33","unstructured":"Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. arXiv preprint arXiv:1509.01626 (2015)"},{"key":"428_CR34","doi-asserted-by":"crossref","unstructured":"Zhang, X., Xu, Y., Lin, X., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et\u00a0al.: Robust log-based anomaly detection on unstable log data. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 807\u2013817 (2019)","DOI":"10.1145\/3338906.3338931"},{"key":"428_CR35","doi-asserted-by":"crossref","unstructured":"Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., Lyu, M.R.: Tools and benchmarks for automated log parsing. In: 2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pp. 121\u2013130. IEEE (2019)","DOI":"10.1109\/ICSE-SEIP.2019.00021"}],"container-title":["Automated Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10515-024-00428-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10515-024-00428-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10515-024-00428-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T15:25:59Z","timestamp":1730129159000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10515-024-00428-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,16]]},"references-count":35,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["428"],"URL":"https:\/\/doi.org\/10.1007\/s10515-024-00428-x","relation":{},"ISSN":["0928-8910","1573-7535"],"issn-type":[{"value":"0928-8910","type":"print"},{"value":"1573-7535","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,16]]},"assertion":[{"value":"30 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"37"}}