{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:05:45Z","timestamp":1776441945061,"version":"3.51.2"},"reference-count":45,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,3,9]],"date-time":"2023-03-09T00:00:00Z","timestamp":1678320000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Anomaly detection is one of the basic issues in data processing that addresses different problems in healthcare sensory data. Technology has made it easier to collect large and highly variant time series data; however, complex predictive analysis models are required to ensure consistency and reliability. With the rise in the size and dimensionality of collected data, deep learning techniques, such as autoencoder (AE), recurrent neural networks (RNN), and long short-term memory (LSTM), have gained more attention and are recognized as state-of-the-art anomaly detection techniques. Recently, developments in transformer-based architecture have been proposed as an improved attention-based knowledge representation scheme. We present an unsupervised transformer-based method to evaluate and detect anomalies in electrocardiogram (ECG) signals. The model architecture comprises two parts: an embedding layer and a standard transformer encoder. We introduce, implement, test, and validate our model in two well-known datasets: ECG5000 and MIT-BIH Arrhythmia. Anomalies are detected based on loss function results between real and predicted ECG time series sequences. We found that the use of a transformer encoder as an alternative model for anomaly detection enables better performance in ECG time series data. The suggested model has a remarkable ability to detect anomalies in ECG signal and outperforms deep learning approaches found in the literature on both datasets. In the ECG5000 dataset, the model can detect anomalies with 99% accuracy, 99% F1-score, 99% AUC score, 98.1% recall, and 100% precision. In the MIT-BIH Arrhythmia dataset, the model achieved an accuracy of 89.5%, F1 score of 92.3%, AUC score of 93%, recall of 98.2%, and precision of 87.1%.<\/jats:p>","DOI":"10.3390\/a16030152","type":"journal-article","created":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T01:31:41Z","timestamp":1678411901000},"page":"152","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":59,"title":["Unsupervised Transformer-Based Anomaly Detection in ECG Signals"],"prefix":"10.3390","volume":"16","author":[{"given":"Abrar","family":"Alamr","sequence":"first","affiliation":[{"name":"Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Abdelmonim","family":"Artoli","sequence":"additional","affiliation":[{"name":"Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"100568","DOI":"10.1016\/j.iot.2022.100568","article-title":"IoT anomaly detection methods and applications: A survey","volume":"19","author":"Chatterjee","year":"2022","journal-title":"Internet Things"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, H., and Boulanger, P. (2022). Structural Anomalies Detection from Electrocardiogram (ECG) with Spectrogram and Handcrafted Features. Sensors, 22.","DOI":"10.3390\/s22072467"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1779","DOI":"10.14778\/3538598.3538602","article-title":"Anomaly Detection in Time Series: A Comprehensive Evaluation","volume":"15","author":"Schmidl","year":"2022","journal-title":"Proc. VLDB Endow."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mehrotra, K.G., Mohan, C.K., and Huang, H. (2017). Introduction. Anomaly Detection Principles and Algorithms, Springer. TSC.","DOI":"10.1007\/978-3-319-67526-8"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1662","DOI":"10.1109\/ACCESS.2017.2779939","article-title":"LSTM Fully Convolutional Networks for Time Series Classification","volume":"6","author":"Karim","year":"2017","journal-title":"IEEE Access"},{"key":"ref_6","first-page":"8","article-title":"Clustering-based real-time anomaly detection\u2014A breakthrough in big data technologies","volume":"33","year":"2022","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1186\/s40537-020-00320-x","article-title":"A comprehensive survey of anomaly detection techniques for high dimensional big data","volume":"7","author":"Thudumu","year":"2020","journal-title":"J. Big Data"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.1109\/TPWRS.2002.804943","article-title":"ARIMA models to predict next-day electricity prices","volume":"18","author":"Contreras","year":"2003","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Gao, J., Liang, F., Fan, W., Wang, C., Sun, Y., and Hann, J. (2010, January 25\u201328). On community outliers and their efficient detection in information networks. Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.","DOI":"10.1145\/1835804.1835907"},{"key":"ref_10","unstructured":"Barnett, V., and Lewis, T. (1984). Applied Probability and Statistics, Wiley. [2nd ed.]."},{"key":"ref_11","unstructured":"Audibert, J., Michiardi, P., Guyard, F., Marti, S., and Zuluaga, M.A. (2020, January 6\u201310). USAD: UnSupervised Anomaly Detection on Multivariate Time Series. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Virtual."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kaur, H., Singh, G., and Minhas, J. (2013). A review of machine learning based anomaly detection techniques. arXiv.","DOI":"10.7753\/IJCATR0202.1020"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/B978-0-12-398537-8.00004-3","article-title":"Anomaly Detection","volume":"14","author":"Laxhammar","year":"2014","journal-title":"Conform. Predict. Reliab. Mach. Learn. Theory Adapt. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly detection: A survy","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surv."},{"key":"ref_15","unstructured":"Ejay, N., Oluwarotimi, W.S., Mojisola, G.A., and Guanglin, L. (2022, January 7\u20139). Intelligence Combiner: A Combination of Deep Learning and Handcrafted Features for an Adolescent Psychosis Prediction using EEG Signals. Proceedings of the 2022 IEEE International Workshop on Metrology for Industry 4.0 & IoT (MetroInd4.0&IoT), Trento, Italy."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, Z., Yeo, C.K., Lee, B.S., and Lau, C.T. (2018, January 17\u201320). Autoencoder-based network anomaly detection. Proceedings of the Wireless Telecommunications Symposium, Phoenix, AZ, USA.","DOI":"10.1109\/WTS.2018.8363930"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nanduri, A., and Sherry, L. (2016, January 19\u201321). Anomaly detection in aircraft data using Recurrent Neural Networks (RNN). Proceedings of the ICNS 2016: Securing an Integrated CNS System to Meet Future Challenges, Herndon, VA, USA.","DOI":"10.1109\/ICNSURV.2016.7486356"},{"key":"ref_18","unstructured":"Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., and Shroff, G. (2016). LSTM-based Encoder-Decoder for Multi-Sensor Anomaly Detection. arXiv."},{"key":"ref_19","unstructured":"Lu, L., Krause, B., Murray, I., and Renals, S. (2017). Multiplicative LSTM for sequence modelling. arXiv."},{"key":"ref_20","unstructured":"Vaswani, I.P.A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., and Kaiser, L. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chauhan, S., and Vig, L. (2015, January 19\u201321). Anomaly detection in ECG time signals via deep long short-term memory networks. Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015, Paris, France.","DOI":"10.1109\/DSAA.2015.7344872"},{"key":"ref_22","first-page":"35","article-title":"Deep learning-based detection of periodic abnormal waves in ECG data","volume":"2233","author":"Sugimoto","year":"2018","journal-title":"Lect. Notes Eng. Comput. Sci."},{"key":"ref_23","unstructured":"Malhotra, P., Vig, L., Shroff, G., and Agarwal, P. (2015, January 22\u201323). Long Short Term Memory networks for anomaly detection in time series. Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2015, Bruges, Belgium."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhu, G., Zhao, H., Liu, H., and Sun, H. (2019, January 25\u201327). A Novel LSTM-GAN Algorithm for Time Series Anomaly Detection. Proceedings of the 2019 Prognostics and System Health Management Conference, PHM-Qingdao 2019, Qingdao, China.","DOI":"10.1109\/PHM-Qingdao46334.2019.8942842"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Schlegl, T., Seeb\u00f6ck, P., Waldstein, S.M., Schmidt-Erfurth, U., and Langs, G. (2017). Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery, Springer.","DOI":"10.1007\/978-3-319-59050-9_12"},{"key":"ref_26","unstructured":"Thill, M., D\u00e4ubener, S., Konen, W., and B\u00e4ck, T. (2019, January 20\u201324). Anomaly Detection in Electrocardiogram Readings with Stacked LSTM Networks. Proceedings of the 19th Conference Information Technologies\u2014Applications and Theory (ITAT 2019), Donovaly, Slovakia."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Xu, L. (2022). TGAN-AD: Transformer-Based GAN for Anomaly Detection of Time Series Data. Appl. Sci., 12.","DOI":"10.3390\/app12168085"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"9179","DOI":"10.1109\/JIOT.2021.3100509","article-title":"Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT","volume":"9","author":"Chen","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"105325","DOI":"10.1016\/j.compbiomed.2022.105325","article-title":"A transformer-based deep neural network for arrhythmia detection using continuous ECG signals","volume":"144","author":"Rui","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"PhysioBank PhysioToolkit (2000). PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101, 215\u2013220.","DOI":"10.1161\/01.CIR.101.23.e215"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/51.932724","article-title":"The impact of the MIT-BIH arrhythmia database","volume":"20","author":"Moody","year":"2001","journal-title":"IEEE Eng. Med. Biol. Mag."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1622","DOI":"10.1007\/s10618-014-0388-4","article-title":"A general framework for never-ending learning from time series streams","volume":"29","author":"Chen","year":"2015","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.cmpb.2015.12.008","article-title":"ECG-based heartbeat classification for arrhythmia detection: A survey","volume":"127","author":"Luz","year":"2016","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"101690","DOI":"10.1016\/j.bspc.2019.101690","article-title":"Heartbeat classification using local transform pattern feature and hybrid neural fuzzy-logic system based on self-organizing map","volume":"57","author":"Lee","year":"2020","journal-title":"Biomed. Signal Process. Control."},{"key":"ref_35","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). European Conference on Computer Vision, Springer."},{"key":"ref_36","unstructured":"Lei Ba, J., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"012049","DOI":"10.1088\/1757-899X\/324\/1\/012049","article-title":"Analysis of the mean absolute error (MAE) and the root mean square error (RMSE) in assessing rounding model","volume":"324","author":"Wang","year":"2018","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_38","unstructured":"Van Rossum, G., and Drake, F.L. (2009). Python 3 Reference Manual:(Python Documentation Manual Part 2), CreateSpace."},{"key":"ref_39","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv."},{"key":"ref_40","unstructured":"Chollet, F. (2015, January 12). Keras. Available online: https:\/\/github.com\/fchollet\/keras."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Pereira, J., and Silveira, M. (March, January 27). Learning Representations from Healthcare Time Series Data for Unsupervised Anomaly Detection. Proceedings of the 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019, Kyoto, Japan.","DOI":"10.1109\/BIGCOMP.2019.8679157"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Matias, P., Folgado, D., Gamboa, H., and Carreiro, A.V. (2021, January 11\u201313). Robust anomaly detection in time series through variational AutoEncoders and a local similarity score. Proceedings of the BIOSIGNALS 2021\u201414th International Conference on Bio-Inspired Systems and Signal Processing; Part of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2021, Virtual.","DOI":"10.5220\/0010320500910102"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Oluwasanmi, A., Aftab, M.U., Baagyere, E., Qin, Z., Ahmad, M., and Mazzara, M. (2021). Attention Autoencoder for Generative Latent Representational Learning in Anomaly Detection. Sensors, 22.","DOI":"10.3390\/s22010123"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1504\/IJCAT.2021.117277","article-title":"MED-NET: A novel approach to ECG anomaly detection using LSTM auto-encoders","volume":"65","author":"Khandual","year":"2021","journal-title":"Int. J. Comput. Appl. Technol."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/TBCAS.2021.3137646","article-title":"ANNet: A Lightweight Neural Network for ECG Anomaly Detection in IoT Edge Sensors","volume":"16","author":"Sivapalan","year":"2022","journal-title":"IEEE Trans. Biomed. Circuits Syst."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/3\/152\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:51:59Z","timestamp":1760122319000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/3\/152"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,9]]},"references-count":45,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["a16030152"],"URL":"https:\/\/doi.org\/10.3390\/a16030152","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,9]]}}}