{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T11:54:38Z","timestamp":1768996478985,"version":"3.49.0"},"reference-count":49,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2019,8,5]],"date-time":"2019-08-05T00:00:00Z","timestamp":1564963200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Labeling activity data is a central part of the design and evaluation of human activity recognition systems. The performance of the systems greatly depends on the quantity and \u201cquality\u201d of annotations; therefore, it is inevitable to rely on users and to keep them motivated to provide activity labels. While mobile and embedded devices are increasingly using deep learning models to infer user context, we propose to exploit on-device deep learning inference using a long short-term memory (LSTM)-based method to alleviate the labeling effort and ground truth data collection in activity recognition systems using smartphone sensors. The novel idea behind this is that estimated activities are used as feedback for motivating users to collect accurate activity labels. To enable us to perform evaluations, we conduct the experiments with two conditional methods. We compare the proposed method showing estimated activities using on-device deep learning inference with the traditional method showing sentences without estimated activities through smartphone notifications. By evaluating with the dataset gathered, the results show our proposed method has improvements in both data quality (i.e., the performance of a classification model) and data quantity (i.e., the number of data collected) that reflect our method could improve activity data collection, which can enhance human activity recognition systems. We discuss the results, limitations, challenges, and implications for on-device deep learning inference that support activity data collection. Also, we publish the preliminary dataset collected to the research community for activity recognition.<\/jats:p>","DOI":"10.3390\/s19153434","type":"journal-article","created":{"date-parts":[[2019,8,5]],"date-time":"2019-08-05T11:17:47Z","timestamp":1565003867000},"page":"3434","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["On-Device Deep Learning Inference for Efficient Activity Data Collection"],"prefix":"10.3390","volume":"19","author":[{"given":"Nattaya","family":"Mairittha","sequence":"first","affiliation":[{"name":"Graduate School of Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka 804-8550, Japan"}]},{"given":"Tittaya","family":"Mairittha","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka 804-8550, Japan"}]},{"given":"Sozo","family":"Inoue","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka 804-8550, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2019,8,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1145\/2499621","article-title":"A tutorial on human activity recognition using body-worn inertial sensors","volume":"46","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bao, L., and Intille, S.S. (2004, January 21\u201323). Activity recognition from user-annotated acceleration data. Proceedings of the International Conference on Pervasive Computing, Linz\/Vienna, Austria.","DOI":"10.1007\/978-3-540-24646-6_1"},{"key":"ref_3","unstructured":"Mairittha, N., and Inoue, S. (March, January 28). Gamification for High-Quality Dataset in Mobile Activity Recognition. Proceedings of the International Conference on Mobile Computing, Applications, and Services, Osaka, Japan."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3098","DOI":"10.1109\/TKDE.2016.2592527","article-title":"Scalable daily human behavioral pattern mining from multivariate temporal data","volume":"28","author":"Rawassizadeh","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1109\/TMM.2013.2265674","article-title":"On-device mobile visual location recognition by integrating vision and inertial sensors","volume":"15","author":"Guan","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"F\u00f6ckler, P., Zeidler, T., Brombach, B., Bruns, E., and Bimber, O. (2005, January 8\u201310). PhoneGuide: Museum guidance supported by on-device object recognition on mobile phones. Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia, Christchurch, New Zealand.","DOI":"10.1145\/1149488.1149490"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1109\/MNET.2018.1700202","article-title":"Learning IoT in edge: Deep learning for the Internet of Things with edge computing","volume":"32","author":"Li","year":"2018","journal-title":"IEEE Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MCE.2016.2640698","article-title":"Deep Learning for Consumer Devices and Services: Pushing the limits for machine learning, artificial intelligence, and computer vision","volume":"6","author":"Lemley","year":"2017","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_10","unstructured":"Lite, T. (2019, August 02). Available online: https:\/\/www.tensorflow.org\/lite."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F., and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1145\/1964897.1964918","article-title":"Activity recognition using cell phone accelerometers","volume":"12","author":"Kwapisz","year":"2011","journal-title":"ACM SigKDD Explor. Newsl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.3390\/s150102059","article-title":"A survey of online activity recognition using mobile phones","volume":"15","author":"Shoaib","year":"2015","journal-title":"Sensors"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/TITB.2007.899496","article-title":"Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions","volume":"12","author":"Ermes","year":"2008","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s12668-013-0088-3","article-title":"A review and taxonomy of activity recognition on mobile phones","volume":"3","author":"Incel","year":"2013","journal-title":"BioNanoScience"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Inoue, S., Ueda, N., Nohara, Y., and Nakashima, N. (2015, January 7\u201311). Mobile activity recognition for a whole day: Recognizing real nursing activities with big dataset. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan.","DOI":"10.1145\/2750858.2807533"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"88","DOI":"10.3390\/computers2020088","article-title":"A review on video-based human activity recognition","volume":"2","author":"Ke","year":"2013","journal-title":"Computers"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cruciani, F., Cleland, I., Nugent, C., McCullagh, P., Synnes, K., and Hallberg, J. (2018). Automatic annotation for human activity recognition in free living using a smartphone. Sensors, 18.","DOI":"10.3390\/s18072203"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yordanova, K., and Kr\u00fcger, F. (2018). Creating and Exploring Semantic Annotation for Behaviour Analysis. Sensors, 18.","DOI":"10.3390\/s18092778"},{"key":"ref_20","unstructured":"Yordanova, K., Paiement, A., Schr\u00f6der, M., Tonkin, E., Woznowski, P., Olsson, C.M., Rafferty, J., and Sztyler, T. (2018). Challenges in annotation of useR data for UbiquitOUs systems: Results from the 1st ARDUOUS workshop. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/MPRV.2018.011591063","article-title":"NoCloud: Exploring network disconnection through on-device data analysis","volume":"17","author":"Rawassizadeh","year":"2018","journal-title":"IEEE Pervasive Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1007\/s11036-012-0368-0","article-title":"A survey of computation offloading for mobile systems","volume":"18","author":"Kumar","year":"2013","journal-title":"Mob. Netw. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cuervo, E., Balasubramanian, A., Cho, D.K., Wolman, A., Saroiu, S., Chandra, R., and Bahl, P. (2010, January 15\u201318). MAUI: Making smartphones last longer with code offload. Proceedings of the 8th International Conference on Mobile Systems, Applications, and Services, San Francisco, CA, USA.","DOI":"10.1145\/1814433.1814441"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/JIOT.2016.2579198","article-title":"Edge computing: Vision and challenges","volume":"3","author":"Shi","year":"2016","journal-title":"IEEE Internet Things J."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yao, S., Hu, S., Zhao, Y., Zhang, A., and Abdelzaher, T. (2017, January 3\u20137). Deepsense: A unified deep learning framework for time-series mobile sensing data processing. Proceedings of the 26th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, Perth, Australia.","DOI":"10.1145\/3038912.3052577"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Okita, T., and Inoue, S. (2018, January 8\u201312). Activity Recognition: Translation across Sensor Modalities Using Deep Learning. Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore.","DOI":"10.1145\/3267305.3267512"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, Y., Zhong, K., Zhang, J., Sun, Q., and Zhao, X. (2016, January 24\u201325). Lstm networks for mobile human activity recognition. Proceedings of the 2016 International Conference on Artificial Intelligence: Technologies and Applications, Bangkok, Thailand.","DOI":"10.2991\/icaita-16.2016.13"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Okita, T., and Inoue, S. (2017, January 11\u201315). Recognition of multiple overlapping activities using compositional CNN-LSTM model. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, HI, USA.","DOI":"10.1145\/3123024.3123095"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alzantot, M., Wang, Y., Ren, Z., and Srivastava, M.B. (2017, January 23). Rstensorflow: Gpu enabled tensorflow for deep learning on commodity android devices. Proceedings of the 1st International Workshop on Deep Learning for Mobile Systems and Applications, Niagara Falls, NY, USA.","DOI":"10.1145\/3089801.3089805"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Cao, Q., Balasubramanian, N., and Balasubramanian, A. (2017, January 23). MobiRNN: Efficient recurrent neural network execution on mobile GPU. Proceedings of the 1st International Workshop on Deep Learning for Mobile Systems and Applications, Niagara Falls, NY, USA.","DOI":"10.1145\/3089801.3089804"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A.R., and Hinton, G. (2013, January 26\u201331). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Schein, A.I., Popescul, A., Ungar, L.H., and Pennock, D.M. (2002, January 11\u201315). Methods and metrics for cold-start recommendations. Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland.","DOI":"10.1145\/564376.564421"},{"key":"ref_33","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s13748-016-0094-0","article-title":"Learning from imbalanced data: Open challenges and future directions","volume":"5","author":"Krawczyk","year":"2016","journal-title":"Prog. Artif. Intell."},{"key":"ref_35","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mairittha, N., Mairittha, T., and Inoue, S. (2018, January 8\u201312). A Mobile App for Nursing Activity Recognition. Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore.","DOI":"10.1145\/3267305.3267633"},{"key":"ref_37","unstructured":"Mairittha, N., and Inoue, S. (June, January 30). Crowdsourcing System Management for Activity Data with Mobile Sensors. Proceedings of the International Conference on Activity and Behavior Computing, Spokane, WA, USA."},{"key":"ref_38","unstructured":"Inoue, S., Mairittha, N., Mairittha, T., and Hossain, T. (June, January 30). Integrating Activity Recognition and Nursing Care Records: The System, Experiment, and the Dataset. Proceedings of the International Conference on Activity and Behavior Computing, Spokane, WA, USA."},{"key":"ref_39","unstructured":"Mairittha, N., Inoue, S., and Mairittha, T. (2019, August 02). FahLog: A Manual Activity Annotation App; Fukuoka, Japan. Available online: https:\/\/play.google.com\/store\/apps\/details?id=jp.sozolab.fahlog&hl=en."},{"key":"ref_40","unstructured":"Fah Sozolab (2019, August 02). Fahact: An Activity Recognition System. Available online: https:\/\/fahact.sozolab.jp."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied Logistic Regression, John Wiley & Sons.","DOI":"10.1002\/9781118548387"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"69","DOI":"10.2307\/2529937","article-title":"Discriminant analysis","volume":"35","author":"Lachenbruch","year":"1979","journal-title":"Biometrics"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/TSMC.1985.6313426","article-title":"A fuzzy k-nearest neighbor algorithm","volume":"SMC-15","author":"Keller","year":"1985","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1109\/21.97458","article-title":"A survey of decision tree classifier methodology","volume":"21","author":"Safavian","year":"1991","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_46","unstructured":"Rish, I. (2001, January 4\u20136). An empirical study of the naive Bayes classifier. Proceedings of the IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, St. John\u2019s, NL, Canada."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1023\/A:1018628609742","article-title":"Least squares support vector machine classifiers","volume":"9","author":"Suykens","year":"1999","journal-title":"Neural Process. Lett."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Breiman, L. (2017). Classification and Regression Trees, Routledge.","DOI":"10.1201\/9781315139470"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"\u00c7\u00fcr\u00fcko\u011flu, N., and \u00d6zyildirim, B.M. (2018, January 4\u20136). Deep Learning on Mobile Systems. Proceedings of the 2018 Innovations in Intelligent Systems and Applications Conference (ASYU), Adana, Turkey.","DOI":"10.1109\/ASYU.2018.8554039"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/15\/3434\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:08:49Z","timestamp":1760188129000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/15\/3434"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,5]]},"references-count":49,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2019,8]]}},"alternative-id":["s19153434"],"URL":"https:\/\/doi.org\/10.3390\/s19153434","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,8,5]]}}}