{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T12:56:08Z","timestamp":1777380968180,"version":"3.51.4"},"reference-count":69,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AIS"],"published-print":{"date-parts":[[2024,9,24]]},"abstract":"<jats:p>We propose a framework for predicting sensor event sequences (SES) in smart homes, which can proactively support residents\u2019 activities and alert them if activities are not completed as intended. We leverage ongoing activity recognition to enhance the prediction performance, employing a GPT2-based model typically used for sentence generation. We hypothesize that the relationship between ongoing activities and SES patterns is akin to the relationship between topics and word sequence patterns in natural language processing (NLP), enabling us to apply the GPT2-based model to SES prediction. We empirically evaluated our method using two real-world datasets in which residents performed their usual daily activities. Our experimental results demonstrates that the use of the GPT2-based model significantly improves the F1 value of SES prediction from 0.461 to 0.708 compared to the state-of-the-art method, and that leveraging knowledge on ongoing activity can further improve performance to\u00a00.837. Achieving these SES predictions using the ongoing activity recognition model required simple feature engineering and modeling, yielding a performance rate of approximately 80%.<\/jats:p>","DOI":"10.3233\/ais-230429","type":"journal-article","created":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T11:57:55Z","timestamp":1711454275000},"page":"275-308","source":"Crossref","is-referenced-by-count":1,"title":["Sensor event sequence prediction for proactive smart home: A GPT2-based autoregressive language model approach"],"prefix":"10.1177","volume":"16","author":[{"given":"Naoto","family":"Takeda","sequence":"first","affiliation":[{"name":"KDDI Research, Inc., Fujimino, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roberto","family":"Legaspi","sequence":"additional","affiliation":[{"name":"KDDI Research, Inc., Fujimino, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yasutaka","family":"Nishimura","sequence":"additional","affiliation":[{"name":"KDDI Research, Inc., Fujimino, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kazushi","family":"Ikeda","sequence":"additional","affiliation":[{"name":"KDDI Research, Inc., Fujimino, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Atsunori","family":"Minamikawa","sequence":"additional","affiliation":[{"name":"KDDI Research, Inc., Fujimino, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Pl\u00f6tz","sequence":"additional","affiliation":[{"name":"School of Interactive Computing, College of Computing, Georgia Institute of Technology, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sonia","family":"Chernova","sequence":"additional","affiliation":[{"name":"School of Interactive Computing, College of Computing, Georgia Institute of Technology, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/AIS-230429_ref1","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.inffus.2018.06.002","article-title":"Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions","volume":"46","author":"Nweke","year":"2019","journal-title":"Information Fusion"},{"issue":"6","key":"10.3233\/AIS-230429_ref2","doi-asserted-by":"publisher","first-page":"725","DOI":"10.3233\/AIS-170460","article-title":"Private assisted house for smart living","volume":"9","author":"Corradini","year":"2017","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"1","key":"10.3233\/AIS-230429_ref3","doi-asserted-by":"publisher","first-page":"49","DOI":"10.3233\/AIS-170471","article-title":"User needs and preferences on AAL systems that support older adults and their carers","volume":"10","author":"Cesta","year":"2018","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"2","key":"10.3233\/AIS-230429_ref4","doi-asserted-by":"publisher","first-page":"103","DOI":"10.3233\/AIS-180484","article-title":"Living preference modeling of smart homes for different target groups","volume":"10","author":"Allameh","year":"2018","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"6","key":"10.3233\/AIS-230429_ref5","doi-asserted-by":"publisher","first-page":"445","DOI":"10.3233\/AIS-180505","article-title":"Towards context-aware assistive applications for aging in place via real-life-proof activity detection","volume":"10","author":"Caroux","year":"2018","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"15","key":"10.3233\/AIS-230429_ref6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s20154227","article-title":"Hardware for recognition of human activities: A review of smart home and AAL related technologies","volume":"20","author":"Sanchez-Comas","year":"2020","journal-title":"Sensors"},{"key":"10.3233\/AIS-230429_ref7","doi-asserted-by":"publisher","DOI":"10.1109\/IS.2018.8710467"},{"key":"10.3233\/AIS-230429_ref8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TSMC.2021.3116141","article-title":"On predicting sensor readings with sequence modeling and reinforcement learning for energy-efficient IoT applications","author":"Laidi","year":"2021","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"10.3233\/AIS-230429_ref9","doi-asserted-by":"publisher","first-page":"111012","DOI":"10.1109\/ACCESS.2019.2933994","article-title":"Predicting sensor events, activities, and time of occurrence using binary sensor data from homes with older adults","volume":"7","author":"Casagrande","year":"2019","journal-title":"IEEE Access"},{"key":"10.3233\/AIS-230429_ref10","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"10.3233\/AIS-230429_ref12","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3495883"},{"key":"10.3233\/AIS-230429_ref13","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774"},{"key":"10.3233\/AIS-230429_ref14","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454804"},{"key":"10.3233\/AIS-230429_ref15","doi-asserted-by":"publisher","DOI":"10.15439\/2020F20"},{"key":"10.3233\/AIS-230429_ref16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67585-5_43"},{"issue":"1","key":"10.3233\/AIS-230429_ref17","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/MIS.2010.112","article-title":"Learning setting-generalized activity models for smart spaces","volume":"27","author":"Cook","year":"2012","journal-title":"IEEE Intelligent Systems"},{"issue":"8","key":"10.3233\/AIS-230429_ref18","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"10.3233\/AIS-230429_ref19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3414\/ME0592","article-title":"Assessing the quality of activities in a smart environment","volume":"48","author":"Cook","year":"2009","journal-title":"Methods of information in medicine"},{"issue":"66","key":"10.3233\/AIS-230429_ref20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42979-020-0070-4","article-title":"Literature review on transfer learning for human activity recognition using mobile and wearable devices with environmental technology","volume":"1","author":"Hernandez","year":"2020","journal-title":"SN Computer Science"},{"key":"10.3233\/AIS-230429_ref21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/widm.1254","article-title":"Recent trends in machine learning for human activity recognition\u00a0\u2013 a survey","volume":"8","author":"Ramamurthy","year":"2018","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"issue":"9","key":"10.3233\/AIS-230429_ref22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s20092702","article-title":"Unsupervised human activity recognition using the clustering approach: A review","volume":"20","author":"Ariza Colpas","year":"2020","journal-title":"Sensors"},{"issue":"4","key":"10.3233\/AIS-230429_ref23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3447744","article-title":"Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities","volume":"54","author":"Chen","year":"2021","journal-title":"ACM Computing Surveys"},{"issue":"5","key":"10.3233\/AIS-230429_ref24","doi-asserted-by":"publisher","first-page":"453","DOI":"10.3233\/AIS-190536","article-title":"Personalized real-time anomaly detection and health feedback for older adults","volume":"11","author":"Parvin","year":"2019","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"key":"10.3233\/AIS-230429_ref25","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2806333"},{"key":"10.3233\/AIS-230429_ref26","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.105569"},{"key":"10.3233\/AIS-230429_ref27","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/461"},{"key":"10.3233\/AIS-230429_ref28","doi-asserted-by":"publisher","first-page":"2567","DOI":"10.1007\/s42835-019-00278-8","article-title":"Vision-based human activity recognition system using depth silhouettes: A smart home system for monitoring the residents","volume":"14","author":"Kim","year":"2019","journal-title":"Journal of Electrical Engineering & Technology"},{"key":"10.3233\/AIS-230429_ref29","doi-asserted-by":"publisher","first-page":"2567","DOI":"10.1007\/s11042-020-09004-3","article-title":"Vision-based human activity recognition: A survey","volume":"79","author":"Romaissa","year":"2020","journal-title":"Multimedia Tools and Applications"},{"issue":"1","key":"10.3233\/AIS-230429_ref30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3448090","article-title":"Ok Google, what am I doing? Acoustic activity recognition bounded by conversational assistant interactions","volume":"5","author":"Adaimi","year":"2021","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"10.3233\/AIS-230429_ref31","unstructured":"G.\u00a0Singla, D.\u00a0Cook and M.\u00a0Schmitter-Edgecombe, Incorporating temporal reasoning into activity recognition for smart home residents, in: Proceedings of the AAAI Workshop on Spatial and Temporal Reasoning, 2008, pp.\u00a053\u201361."},{"key":"10.3233\/AIS-230429_ref32","doi-asserted-by":"publisher","DOI":"10.1145\/1409635.1409637"},{"key":"10.3233\/AIS-230429_ref33","doi-asserted-by":"publisher","first-page":"299","DOI":"10.18178\/ijmlc.2020.10.2.935","article-title":"Investigating best approaches for activity classification in a fully instrumented smarthome environment","volume":"10","author":"Mihoub","year":"2020","journal-title":"International Journal of Machine Learning and Computing"},{"issue":"5","key":"10.3233\/AIS-230429_ref34","doi-asserted-by":"publisher","first-page":"393","DOI":"10.3233\/AIS-180496","article-title":"A probabilistic data-driven method for human activity recognition","volume":"10","author":"Foudeh","year":"2018","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"4","key":"10.3233\/AIS-230429_ref35","doi-asserted-by":"publisher","first-page":"301","DOI":"10.3233\/AIS-190529","article-title":"Sensor-based activity recognition in the context of ambient assisted living systems: A review","volume":"11","author":"Patel","year":"2019","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"3","key":"10.3233\/AIS-230429_ref36","doi-asserted-by":"publisher","first-page":"183","DOI":"10.3233\/AIS-200562","article-title":"Using continuous sensor data to formalize a model of in-home activity patterns","volume":"12","author":"Lin","year":"2020","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"6","key":"10.3233\/AIS-230429_ref37","doi-asserted-by":"publisher","first-page":"495","DOI":"10.3233\/AIS-190541","article-title":"Representation learning for minority and subtle activities in a smart home environment","volume":"11","author":"Rosales Sanabria","year":"2019","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"key":"10.3233\/AIS-230429_ref38","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.214"},{"key":"10.3233\/AIS-230429_ref39","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.artmed.2015.12.001","article-title":"SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment","volume":"67","author":"Riboni","year":"2016","journal-title":"Artificial Intelligence in Medicine"},{"key":"10.3233\/AIS-230429_ref40","doi-asserted-by":"publisher","DOI":"10.1109\/CcS49175.2020.9231478"},{"key":"10.3233\/AIS-230429_ref41","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1301.3781"},{"issue":"20","key":"10.3233\/AIS-230429_ref42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/electronics10202498","article-title":"Using language model to bootstrap human activity recognition ambient sensors based in smart homes","volume":"10","author":"Bouchabou","year":"2021","journal-title":"Electronics"},{"key":"10.3233\/AIS-230429_ref43","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"10.3233\/AIS-230429_ref44","doi-asserted-by":"publisher","DOI":"10.5555\/1251480.1251493"},{"key":"10.3233\/AIS-230429_ref45","unstructured":"A.\u00a0Radford and K.\u00a0Narasimhan, Improving Language Understanding by Generative Pre-Training, 2018."},{"key":"10.3233\/AIS-230429_ref46","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.05858"},{"issue":"3","key":"10.3233\/AIS-230429_ref47","doi-asserted-by":"publisher","first-page":"820","DOI":"10.1109\/TSMCB.2012.2216873","article-title":"Activity discovery and activity recognition: A new partnership","volume":"43","author":"Cook","year":"2013","journal-title":"IEEE Transactions on Cybernetics"},{"issue":"6","key":"10.3233\/AIS-230429_ref48","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1109\/TSMCC.2012.2198883","article-title":"Sensor-based activity recognition","volume":"42","author":"Chen","year":"2012","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)"},{"key":"10.3233\/AIS-230429_ref49","doi-asserted-by":"publisher","DOI":"10.5555\/3294996.3295074"},{"key":"10.3233\/AIS-230429_ref50","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1162\/tacl_a_00302","article-title":"A knowledge-enhanced pretraining model for commonsense story generation","volume":"8","author":"Guan","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"10.3233\/AIS-230429_ref51","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.comcom.2021.01.002","article-title":"Singability-enhanced lyric generator with music style transfer","volume":"168","author":"Chang","year":"2021","journal-title":"Computer Communications"},{"key":"10.3233\/AIS-230429_ref52","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1908.04319"},{"key":"10.3233\/AIS-230429_ref53","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.630"},{"key":"10.3233\/AIS-230429_ref54","doi-asserted-by":"publisher","first-page":"993","DOI":"10.5555\/944919.944937","article-title":"Latent Dirichlet allocation","volume":"3","author":"Blei","year":"2003","journal-title":"Journal of Machine Learning Research"},{"issue":"4","key":"10.3233\/AIS-230429_ref55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s19040947","article-title":"A human activity recognition algorithm based on stacking denoising autoencoder and LightGBM","volume":"19","author":"Gao","year":"2019","journal-title":"Sensors"},{"key":"10.3233\/AIS-230429_ref56","doi-asserted-by":"publisher","DOI":"10.1109\/GCWkshps50303.2020.9367452"},{"key":"10.3233\/AIS-230429_ref57","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-53917-6_16"},{"key":"10.3233\/AIS-230429_ref58","doi-asserted-by":"publisher","DOI":"10.1109\/IE.2018.00014"},{"key":"10.3233\/AIS-230429_ref59","doi-asserted-by":"publisher","first-page":"3086","DOI":"10.1007\/s10489-021-02193-w","article-title":"Using artificial intelligence techniques for COVID-19 genome analysis","volume":"51","author":"Nawaz","year":"2021","journal-title":"Applied Intelligence"},{"key":"10.3233\/AIS-230429_ref60","doi-asserted-by":"publisher","DOI":"10.1145\/3267242.3267287"},{"key":"10.3233\/AIS-230429_ref61","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.158"},{"issue":"6014","key":"10.3233\/AIS-230429_ref62","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1126\/science.1199644","article-title":"Quantitative analysis of culture using millions of digitized books","volume":"331","author":"Michel","year":"2011","journal-title":"Science"},{"key":"10.3233\/AIS-230429_ref63","unstructured":"C.-Y.\u00a0Lin, ROUGE: A package for automatic evaluation of summaries, in: Proceedings of the ACL Workshop: Text Summarization Braches Out 2004, 2004, pp.\u00a074\u201381."},{"issue":"5","key":"10.3233\/AIS-230429_ref64","doi-asserted-by":"publisher","first-page":"1077","DOI":"10.1162\/jocn_a_00204","article-title":"Hierarchies in action and motor control","volume":"24","author":"Uithol","year":"2012","journal-title":"Journal of Cognitive Neuroscience"},{"key":"10.3233\/AIS-230429_ref65","doi-asserted-by":"publisher","first-page":"1077","DOI":"10.3389\/fnbot.2014.00013","article-title":"Everyday robotic action: Lessons from human action control","volume":"8","author":"De Kleijn","year":"2014","journal-title":"Frontiers in Neurorobotics"},{"issue":"22","key":"10.3233\/AIS-230429_ref66","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s19225043","article-title":"Characterizing word embeddings for zero-shot sensor-based human activity recognition","volume":"19","author":"Matsuki","year":"2019","journal-title":"Sensors"},{"key":"10.3233\/AIS-230429_ref67","doi-asserted-by":"publisher","DOI":"10.1109\/PerComWorkshops51409.2021.9431141"},{"key":"10.3233\/AIS-230429_ref68","doi-asserted-by":"publisher","DOI":"10.1109\/ICIEAM51226.2021.9446389"},{"issue":"3","key":"10.3233\/AIS-230429_ref69","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3351228","article-title":"Leveraging active learning and conditional mutual information to minimize data annotation in human activity recognition","volume":"3","author":"Adaimi","year":"2019","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"10.3233\/AIS-230429_ref70","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOM.2018.8444590"}],"container-title":["Journal of Ambient Intelligence and Smart Environments"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/AIS-230429","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T09:19:29Z","timestamp":1777367969000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.medra.org\/servlet\/aliasResolver?alias=iospress&doi=10.3233\/AIS-230429"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,24]]},"references-count":69,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/ais-230429","relation":{},"ISSN":["1876-1372","1876-1364"],"issn-type":[{"value":"1876-1372","type":"electronic"},{"value":"1876-1364","type":"print"}],"subject":[],"published":{"date-parts":[[2024,9,24]]}}}