{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T14:29:18Z","timestamp":1768832958652,"version":"3.49.0"},"reference-count":39,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,1,24]],"date-time":"2020-01-24T00:00:00Z","timestamp":1579824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>By developing awareness of smartphone activities that the user is performing on their smartphone, such as scrolling feeds, typing and watching videos, we can develop application features that are beneficial to the users, such as personalization. It is currently not possible to access real-time smartphone activities directly, due to standard smartphone privileges and if internal movement sensors can detect them, there may be implications for access policies. Our research seeks to understand whether the sensor data from existing smartphone inertial measurement unit (IMU) sensors (triaxial accelerometers, gyroscopes and magnetometers) can be used to classify typical human smartphone activities. We designed and conducted a study with human participants which uses an Android app to collect motion data during scrolling, typing and watching videos, while walking or seated and the baseline of smartphone non-use, while sitting and walking. We then trained a machine learning (ML) model to perform real-time activity recognition of those eight states. We investigated various algorithms and parameters for the best accuracy. Our optimal solution achieved an accuracy of 78.6% with the Extremely Randomized Trees algorithm, data sampled at 50 Hz and 5-s windows. We conclude by discussing the viability of using IMU sensors to recognize common smartphone activities.<\/jats:p>","DOI":"10.3390\/s20030655","type":"journal-article","created":{"date-parts":[[2020,1,24]],"date-time":"2020-01-24T11:01:00Z","timestamp":1579863660000},"page":"655","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Real-time Smartphone Activity Classification Using Inertial Sensors\u2014Recognition of Scrolling, Typing, and Watching Videos While Sitting or Walking"],"prefix":"10.3390","volume":"20","author":[{"given":"Sijie","family":"Zhuo","sequence":"first","affiliation":[{"name":"Department of Electrical, Computer and Software Engineering, University of Auckland, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lucas","family":"Sherlock","sequence":"additional","affiliation":[{"name":"Department of Electrical, Computer and Software Engineering, University of Auckland, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7245-0367","authenticated-orcid":false,"given":"Gillian","family":"Dobbie","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Auckland, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yun Sing","family":"Koh","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Auckland, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giovanni","family":"Russello","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Auckland, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5541-4425","authenticated-orcid":false,"given":"Danielle","family":"Lottridge","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Auckland, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,24]]},"reference":[{"key":"ref_1","unstructured":"Hauser, E. (2011). Digital Content Personalization Method and System. (8,019,777), U.S. Patent."},{"key":"ref_2","unstructured":"Linden, G.D., Smith, B.R., and Zada, N.K. (2011). Content Personalization Based on Actions Performed During Browsing Sessions. (7,970,664), U.S. Patent."},{"key":"ref_3","unstructured":"Smith, B.R., Linden, G.D., and Zada, N.K. (2005). Content Personalization Based on Actions Performed During a Current Browsing Session. (6,853,982), U.S. Patent."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1177\/1464884910388228","article-title":"Making \u2018The Daily Me\u2019: Technology, economics and habit in the mainstream assimilation of personalized news","volume":"12","author":"Thurman","year":"2011","journal-title":"Journalism"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Narain, S., Vo-Huu, T.D., Block, K., and Noubir, G. (2016, January 22\u201326). Inferring user routes and locations using zero-permission mobile sensors. Proceedings of the 2016 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.","DOI":"10.1109\/SP.2016.31"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ning, R., Wang, C., Xin, C., Li, J., and Wu, H. (2018, January 19\u201323). Deepmag: Sniffing mobile apps in magnetic field through deep convolutional neural networks. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), Athens, Greece.","DOI":"10.1109\/PERCOM.2018.8444573"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/s13635-017-0061-8","article-title":"Sensor Guardian: Prevent privacy inference on Android sensors","volume":"2017","author":"Bai","year":"2017","journal-title":"EURASIP J. Inf. Secur."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/j.future.2017.11.029","article-title":"A robust human activity recognition system using smartphone sensors and deep learning","volume":"81","author":"Hassan","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Voicu, R.A., Dobre, C., Bajenaru, L., and Ciobanu, R.I. (2019). Human Physical Activity Recognition Using Smartphone Sensors. Sensors, 19.","DOI":"10.3390\/s19030458"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.procs.2014.07.009","article-title":"A study on human activity recognition using accelerometer data from smartphones","volume":"34","author":"Bayat","year":"2014","journal-title":"Procedia Comput. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ustev, Y.E., Durmaz Incel, O., and Ersoy, C. (2013, January 8\u201312). User, device and orientation independent human activity recognition on mobile phones: Challenges and a proposal. Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, Zurich, Switzerland.","DOI":"10.1145\/2494091.2496039"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1145\/1964897.1964918","article-title":"Activity recognition using cell phone accelerometers","volume":"12","author":"Kwapisz","year":"2011","journal-title":"ACM SigKDD Explor. Newsl."},{"key":"ref_14","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24\u201326). A public domain dataset for human activity recognition using smartphones. Proceedings of the Proceedings of the 21th International European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Khan, A.M., Lee, Y.K., Lee, S.Y., and Kim, T.S. (2010, January 21\u201323). Human activity recognition via an accelerometer-enabled-smartphone using kernel discriminant analysis. Proceedings of the 2010 5th International Conference on Future Information Technology, Busan, Korea.","DOI":"10.1109\/FUTURETECH.2010.5482729"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shoaib, M., Bosch, S., Incel, O., Scholten, H., and Havinga, P. (2016). Complex human activity recognition using smartphone and wrist-worn motion sensors. Sensors, 16.","DOI":"10.3390\/s16040426"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"10701","DOI":"10.1007\/s11042-015-3188-y","article-title":"Towards unsupervised physical activity recognition using smartphone accelerometers","volume":"76","author":"Lu","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1109\/34.824823","article-title":"Looking at people: Sensing for ubiquitous and wearable computing","volume":"22","author":"Pentland","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Anjum, A., and Ilyas, M.U. (2013, January 11\u201314). Activity recognition using smartphone sensors. Proceedings of the 2013 IEEE 10th Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA.","DOI":"10.1109\/CCNC.2013.6488584"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s12668-013-0088-3","article-title":"A review and taxonomy of activity recognition on mobile phones","volume":"3","author":"Incel","year":"2013","journal-title":"BioNanoScience"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1109\/TST.2014.6838194","article-title":"Activity recognition with smartphone sensors","volume":"19","author":"Su","year":"2014","journal-title":"Tsinghua Sci. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/MCOM.2010.5560598","article-title":"A survey of mobile phone sensing","volume":"48","author":"Lane","year":"2010","journal-title":"IEEE Commun. Mag."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"LiKamWa, R., Liu, Y., Lane, N.D., and Zhong, L. (2013, January 25\u201328). Moodscope: Building a mood sensor from smartphone usage patterns. In Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services, Taipei, Taiwan.","DOI":"10.1145\/2462456.2483967"},{"key":"ref_24","unstructured":"Gimpel, H., Regal, C., and Schmidt, M. (2015, January 26\u201329). myStress: Unobtrusive Smartphone-Based Stress Detection. Proceedings of the 23th European Conference on Information Systems (ECIS 2015), M\u00fcnster, Germany."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Goel, M., Findlater, L., and Wobbrock, J. (2012, January 5\u201310). WalkType: Using accelerometer data to accomodate situational impairments in mobile touch screen text entry. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA.","DOI":"10.1145\/2207676.2208662"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Marquardt, P., Verma, A., Carter, H., and Traynor, P. (2011, January 22\u201324). (sp) iphone: Decoding vibrations from nearby keyboards using mobile phone accelerometers. Proceedings of the 18th ACM Conference on Computer and Communications Security, Hong Kong, China.","DOI":"10.1145\/2046707.2046771"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e130","DOI":"10.2196\/jmir.2208","article-title":"Classification accuracies of physical activities using smartphone motion sensors","volume":"14","author":"Wu","year":"2012","journal-title":"J. Med Internet Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.3390\/s150102059","article-title":"A survey of online activity recognition using mobile phones","volume":"15","author":"Shoaib","year":"2015","journal-title":"Sensors"},{"key":"ref_29","first-page":"877","article-title":"HMOG: New behavioral biometric features for continuous authentication of smartphone users","volume":"11","author":"Yang","year":"2015","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kumar, R., Phoha, V.V., and Serwadda, A. (2016, January 6\u20139). Continuous authentication of smartphone users by fusing typing, swiping, and phone movement patterns. Proceedings of the 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), Niagara Falls, NY, USA.","DOI":"10.1109\/BTAS.2016.7791164"},{"key":"ref_31","unstructured":"Zhuo, S. (2019). Project Number: 53 Using Smartphone Sensors to Classify User Smartphone Activities. [Honour\u2019s Thesis, University of Auckland]."},{"key":"ref_32","unstructured":"Sherlock, L. (2019). Project Number: 53 Using Smartphone Sensors to Classify User Smartphone Activities. [Honour\u2019s Thesis, University of Auckland]."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"MacKenzie, I.S., and Soukoreff, R.W. (2003, January 5\u201310). Phrase sets for evaluating text entry techniques. Proceedings of the CHI\u201903 Extended Abstracts on Human Factors in Computing Systems, Ft. Lauderdale, FL, USA.","DOI":"10.1145\/765891.765971"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"23168","DOI":"10.3390\/s150923168","article-title":"Performance evaluation of smartphone inertial sensors measurement for range of motion","volume":"15","author":"Mourcou","year":"2015","journal-title":"Sensors"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.tjem.2018.08.001","article-title":"User\u2019s guide to correlation coefficients","volume":"18","author":"Akoglu","year":"2018","journal-title":"Turk. J. Emerg. Med."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2627","DOI":"10.1016\/S1352-2310(97)00447-0","article-title":"Artificial neural networks (the multilayer perceptron)\u2014A review of applications in the atmospheric sciences","volume":"32","author":"Gardner","year":"1998","journal-title":"Atmos. Environ."},{"key":"ref_37","first-page":"18","article-title":"Classification and regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10994-006-6226-1","article-title":"Extremely randomized trees","volume":"63","author":"Geurts","year":"2006","journal-title":"Mach. Learn."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1080\/15213269.2017.1406805","article-title":"The fragmentation of work, entertainment, E-Mail, and news on a personal computer: Motivational predictors of switching between media content","volume":"21","author":"Yeykelis","year":"2018","journal-title":"Media Psychol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/3\/655\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:30:25Z","timestamp":1760362225000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/3\/655"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,24]]},"references-count":39,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["s20030655"],"URL":"https:\/\/doi.org\/10.3390\/s20030655","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,24]]}}}