{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T17:25:04Z","timestamp":1779384304036,"version":"3.53.1"},"reference-count":42,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T00:00:00Z","timestamp":1622160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Continuous authentication systems have been proposed as a promising solution to authenticate users in smartphones in a non-intrusive way. However, current systems have important weaknesses related to the amount of data or time needed to build precise user profiles, together with high rates of false alerts. Voice is a powerful dimension for identifying subjects but its suitability and importance have not been deeply analyzed regarding its inclusion in continuous authentication systems. This work presents the S3 platform, an artificial intelligence-enabled continuous authentication system that combines data from sensors, applications statistics and voice to authenticate users in smartphones. Experiments have tested the relevance of each kind of data, explored different strategies to combine them, and determined how many days of training are needed to obtain good enough profiles. Results showed that voice is much more relevant than sensors and applications statistics when building a precise authenticating system, and the combination of individual models was the best strategy. Finally, the S3 platform reached a good performance with only five days of use available for training the users\u2019 profiles. As an additional contribution, a dataset with 21 volunteers interacting freely with their smartphones for more than sixty days has been created and made available to the community.<\/jats:p>","DOI":"10.3390\/s21113765","type":"journal-article","created":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T03:45:29Z","timestamp":1622432729000},"page":"3765","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["S3: An AI-Enabled User Continuous Authentication for Smartphones Based on Sensors, Statistics and Speaker Information"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6521-7890","authenticated-orcid":false,"given":"Juan Manuel","family":"Esp\u00edn L\u00f3pez","sequence":"first","affiliation":[{"name":"Department of Information and Communications Engineering (DIIC), University of Murcia, 30100 Murcia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7125-1710","authenticated-orcid":false,"given":"Alberto","family":"Huertas Celdr\u00e1n","sequence":"additional","affiliation":[{"name":"Communication Systems Group (CSG), Department of Informatics (IfI), University of Z\u00fcrich UZH, CH-8050 Z\u00fcrich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2742-4607","authenticated-orcid":false,"given":"Javier G.","family":"Mar\u00edn-Bl\u00e1zquez","sequence":"additional","affiliation":[{"name":"Department of Information and Communications Engineering (DIIC), University of Murcia, 30100 Murcia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4550-0183","authenticated-orcid":false,"given":"Francisco","family":"Esquembre","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of Murcia, 30100 Murcia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5532-6604","authenticated-orcid":false,"given":"Gregorio","family":"Mart\u00ednez P\u00e9rez","sequence":"additional","affiliation":[{"name":"Department of Information and Communications Engineering (DIIC), University of Murcia, 30100 Murcia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,28]]},"reference":[{"key":"ref_1","unstructured":"Wang, G., Feng, J., Bhuiyan, M.Z.A., and Lu, R. (2019). Continuous Authentication Using Mouse Clickstream Data Analysis. Security, Privacy, and Anonymity in Computation, Communication, and Storage, Springer International Publishing."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gonzalez-Manzano, L., Fuentes, J.M.D., and Ribagorda, A. (2019). Leveraging User-Related Internet of Things for Continuous Authentication: A Survey. ACM Comput. Surv., 52.","DOI":"10.1145\/3314023"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Jorquera Valero, J., S\u00e1nchez S\u00e1nchez, P., Fern\u00e1ndez Maim\u00f3, L., Huertas Celdr\u00e1n, A., Arjona Fern\u00e1ndez, M., De Los Santos V\u00edlchez, S., and Mart\u00ednez P\u00e9rez, G. (2018). Improving the Security and QoE in Mobile Devices through an Intelligent and Adaptive Continuous Authentication System. Sensors, 18.","DOI":"10.3390\/s18113769"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Feng, T., Zhao, X., DeSalvo, N., Gao, Z., Wang, X., and Shi, W. (2015, January 14\u201316). Security after login: Identity change detection on smartphones using sensor fusion. Proceedings of the 2015 IEEE International Symposium on Technologies for Homeland Security (HST), Waltham, MA, USA.","DOI":"10.1109\/THS.2015.7225268"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Shi, W., Yang, J., Jiang, Y., Yang, F., and Xiong, Y. (2011, January 10\u201312). SenGuard: Passive user identification on smartphones using multiple sensors. Proceedings of the 2011 IEEE 7th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), Shanghai, China.","DOI":"10.1109\/WiMOB.2011.6085412"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shon, S., Dehak, N., Reynolds, D., and Glass, J. (2019). MCE 2018: The 1st Multi-target Speaker Detection and Identification Challenge Evaluation. arXiv.","DOI":"10.21437\/Interspeech.2019-1572"},{"key":"ref_7","unstructured":"Brunet, K., Taam, K., Cherrier, E., Faye, N., and Rosenberger, C. (2013, January 16\u201319). Speaker Recognition for Mobile User Authentication: An Android Solution. Proceedings of the 8\u00e8me Conf\u00e9rence sur la S\u00e9curit\u00e9 des Architectures R\u00e9seaux et Syst\u00e8mes d\u2019Information (SAR SSI), Mont-de-Marsan, France."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.cose.2013.05.005","article-title":"A framework for continuous, transparent mobile device authentication","volume":"39","author":"Crawford","year":"2013","journal-title":"Comput. Secur."},{"key":"ref_9","unstructured":"Esp\u00edn L\u00f3pez, J.M., Huertas Celdr\u00e1n, A., Mar\u00edn-Bl\u00e1zquez, J.G., Esquembre Mart\u00ednez, F., and P\u00e9rez, G.M. (2021, April 07). S3 Dataset. Available online: https:\/\/figshare.com\/articles\/dataset\/S3Dataset_zip\/14410229\/2."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.inffus.2020.08.021","article-title":"Behavioral biometrics & continuous user authentication on mobile devices: A survey","volume":"66","author":"Stylios","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Abuhamad, M., Abusnaina, A., Nyang, D., and Mohaisen, D. (2020). Sensor-Based Continuous Authentication of Smartphones\u2019 Users Using Behavioral Biometrics: A Contemporary Survey. IEEE Internet Things J., 1.","DOI":"10.1109\/JIOT.2020.3020076"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.comcom.2020.09.006","article-title":"A survey on continuous authentication methods in Internet of Things environment","volume":"163","author":"Zagrouba","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3397179","article-title":"SCANet: Sensor-based continuous authentication with two-stream convolutional neural networks","volume":"16","author":"Li","year":"2020","journal-title":"ACM Trans. Sens. Netw. (TOSN)"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Barlas, Y., Basar, O.E., Akan, Y., Isbilen, M., Alptekin, G.I., and Incel, O.D. (2020, January 9\u201311). DAKOTA: Continuous Authentication with Behavioral Biometrics in a Mobile Banking Application. Proceedings of the 2020 5th International Conference on Computer Science and Engineering (UBMK), Diyarbakir, Turkey.","DOI":"10.1109\/UBMK50275.2020.9219365"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.procs.2019.08.027","article-title":"Towards Continuous Authentication on Mobile Phones using Deep Learning Models","volume":"155","author":"Volaka","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bourlai, T., Karampelas, P., and Patel, V.M. (2020). Mobile Active Authentication based on Multiple Biometric and Behavioral Patterns. Securing Social Identity in Mobile Platforms: Technologies for Security, Privacy and Identity Management, Springer International Publishing.","DOI":"10.1007\/978-3-030-39489-9"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Buriro, A., Crispo, B., Gupta, S., and Del Frari, F. (2018, January 19\u201321). DIALERAUTH: A Motion-Assisted Touch-Based Smartphone User Authentication Scheme. Proceedings of the Eighth ACM Conference on Data and Application Security and Privacy; Association for Computing Machinery, CODASPY \u201918, Tempe, AZ, USA.","DOI":"10.1145\/3176258.3176318"},{"key":"ref_18","first-page":"89","article-title":"AnswerAuth: A bimodal behavioral biometric-based user authentication scheme for smartphones","volume":"44","author":"Buriro","year":"2019","journal-title":"J. Inf. Secur. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"102168","DOI":"10.1016\/j.cose.2020.102168","article-title":"AuthCODE: A privacy-preserving and multi-device continuous authentication architecture based on machine and deep learning","volume":"103","year":"2021","journal-title":"Comput. Secur."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"100162","DOI":"10.1016\/j.smhl.2020.100162","article-title":"GaitCode: Gait-based Continuous Authentication Using Multimodal Learning and Wearable Sensors","volume":"19","author":"Papavasileiou","year":"2020","journal-title":"Smart Health"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1109\/TIFS.2015.2506542","article-title":"HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users","volume":"11","author":"Yang","year":"2016","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_22","unstructured":"Yang, Q., Peng, G., Nguyen, D., Qi, X., Zhou, G., Sitov\u00e1, Z., Gasti, P., and Balagani, K. (2021, April 07). H-MOG Data Set: A Multimodal Data Set for Evaluating Continuous Authentication Performance in Smartphones. Available online: http:\/\/www.cs.wm.edu\/~qyang\/hmog.html."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mahbub, U., Sarkar, S., Patel, V.M., and Chellappa, R. (2016, January 6\u20139). Active user authentication for smartphones: A challenge data set and benchmark results. Proceedings of the 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), Niagara Falls, NY, USA.","DOI":"10.1109\/BTAS.2016.7791155"},{"key":"ref_24","unstructured":"Mahbub, U. (2021, April 07). University of Maryland Active Authentication-02 UMDAA-02 Dataset. Available online: https:\/\/umdaa02.github.io\/."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Papamichail, M.D., Chatzidimitriou, K.C., Karanikiotis, T., Oikonomou, N.C.I., Symeonidis, A.L., and Saripalle, S.K. (2019). BrainRun: A Behavioral Biometrics Dataset towards Continuous Implicit Authentication. Data, 4.","DOI":"10.3390\/data4020060"},{"key":"ref_26","unstructured":"Belman, A.K., Wang, L., Iyengar, S.S., Sniatala, P., Wright, R., Dora, R., Baldwin, J., Jin, Z., and Phoha, V.V. (2019). SU-AIS BB-MAS (Syracuse University and Assured Information Security\u2014Behavioral Biometrics Multi-device and multi-Activity data from Same users) Dataset. IEEE DataPort."},{"key":"ref_27","unstructured":"S\u00e1nchez S\u00e1nchez, P.M., Fern\u00e1ndez Maim\u00f3, L., Huertas Celdr\u00e1n, A., and Mart\u00ednez P\u00e9rez, G. (2020). Authcode-Dataset. IEEE DataPort."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1744","DOI":"10.1109\/TNN.2011.2167240","article-title":"Learning Speaker-Specific Characteristics With a Deep Neural Architecture","volume":"22","author":"Chen","year":"2011","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"293","DOI":"10.21437\/Odyssey.2014-44","article-title":"Deep Neural Networks for extracting Baum-Welch statistics for Speaker Recognition","volume":"2014","author":"Kenny","year":"2014","journal-title":"Odyssey"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1671","DOI":"10.1109\/LSP.2015.2420092","article-title":"Deep Neural Network Approaches to Speaker and Language Recognition","volume":"22","author":"Richardson","year":"2015","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., and Khudanpur, S. (2018, January 15\u201320). X-vectors: Robust DNN Embeddings for Speaker Recognition. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8461375"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Prince, S., and Elder, J. (2007, January 14\u201321). Probabilistic Linear Discriminant Analysis for Inferences About Identity. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil.","DOI":"10.1109\/ICCV.2007.4409052"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Huang, C.L. (2020, January 1\u20135). Speaker Characterization Using TDNN, TDNN-LSTM, TDNN-LSTM-Attention based Speaker Embeddings for NIST SRE 2019. Proceedings of the Odyssey 2020 The Speaker and Language Recognition Workshop, Tokyo, Japan.","DOI":"10.21437\/Odyssey.2020-60"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, R., Chen, D., and Zhang, W. (2020, January 4\u20138). Voiceai Systems to NIST Sre19 Evaluation: Robust Speaker Recognition on Conversational Telephone Speech. Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054624"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3820","DOI":"10.21437\/Interspeech.2020-2538","article-title":"Angular Margin Centroid Loss for Text-independent Speaker Recognition","volume":"2020","author":"Wei","year":"2020","journal-title":"Proc. Interspeech 2020"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lu, X., Shen, P., Tsao, Y., and Kawai, H. (2021). Integrating a joint Bayesian generative model in a discriminative learning framework for speaker verification. arXiv.","DOI":"10.1109\/TASLP.2021.3129360"},{"key":"ref_37","unstructured":"Cai, W., and Li, M. (2020). A Unified Deep Speaker Embedding Framework for Mixed-Bandwidth Speech Data. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Khan, U., and Hernando, J. (2021). Self-supervised Deep Learning Approaches to Speaker Recognition: A Ph.D. Thesis Overview. Proc. IberSPEECH, 175\u2013179.","DOI":"10.21437\/IberSPEECH.2021-38"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.jnca.2018.02.020","article-title":"Continuous authentication of smartphone users based on activity pattern recognition using passive mobile sensing","volume":"109","author":"Naeem","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_40","unstructured":"Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., and Schwarz, P. (2011, January 11\u201315). The Kaldi Speech Recognition Toolkit. Proceedings of the IEEE 2011 Workshop on Automatic Speech Recognition and Understanding, Waikoloa, HI, USA. IEEE Catalog No.: CFP11SRW-USB."},{"key":"ref_41","unstructured":"Povey, D. (2021, April 07). Models Kaldi. Available online: https:\/\/kaldi-asr.org\/models.html."},{"key":"ref_42","first-page":"1","article-title":"PyOD: A Python Toolbox for Scalable Outlier Detection","volume":"20","author":"Zhao","year":"2019","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/11\/3765\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:10:08Z","timestamp":1760163008000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/11\/3765"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,28]]},"references-count":42,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["s21113765"],"URL":"https:\/\/doi.org\/10.3390\/s21113765","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,28]]}}}