{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T14:51:55Z","timestamp":1777128715744,"version":"3.51.4"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006374","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62102308"],"award-info":[{"award-number":["62102308"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Initiative Postdocs Supporting Program","award":["BX2021243"],"award-info":[{"award-number":["BX2021243"]}]},{"DOI":"10.13039\/501100006374","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2021M702627"],"award-info":[{"award-number":["2021M702627"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,12,19]]},"abstract":"<jats:p>Enabling smart devices to learn automating actions as expected is a crucial yet challenging task. The traditional Trigger-Action rule approach for device automation is prone to ambiguity in complex scenarios. To address this issue, we propose a data-driven approach that leverages recorded user-driven event sequences to predict potential actions users may take and generate fine-grained device automation sequences. Our key intuition is that user-driven event sequences, like human-written articles and programs, are governed by consistent semantic contexts and contain regularities that can be modeled to generate sequences that express the user's preferences. We introduce ASGen, a deep learning framework that combines sequential information, event attributes, and external knowledge to form the event representation and output sequences of arbitrary length to facilitate automation. To evaluate our approach from both quantitative and qualitative perspectives, we conduct two studies using a realistic dataset containing over 4.4 million events. Our results show that our approach surpasses other methods by providing more accurate recommendations. And the automation sequences generated by our model are perceived as equally or even more rational and useful compared to those generated by humans.<\/jats:p>","DOI":"10.1145\/3631427","type":"journal-article","created":{"date-parts":[[2024,1,12]],"date-time":"2024-01-12T12:52:04Z","timestamp":1705063924000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Learning from User-driven Events to Generate Automation Sequences"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4186-0408","authenticated-orcid":false,"given":"Yunpeng","family":"Song","sequence":"first","affiliation":[{"name":"MOE KLINNS Lab, Xi'an Jiaotong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8628-1751","authenticated-orcid":false,"given":"Yiheng","family":"Bian","sequence":"additional","affiliation":[{"name":"MOE KLINNS Lab, Xi'an Jiaotong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-0593-902X","authenticated-orcid":false,"given":"Xiaorui","family":"Wang","sequence":"additional","affiliation":[{"name":"MOE KLINNS Lab, Xi'an Jiaotong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4903-3992","authenticated-orcid":false,"given":"Zhongmin","family":"Cai","sequence":"additional","affiliation":[{"name":"MOE KLINNS Lab, Xi'an Jiaotong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,1,12]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2023. Home Assistant."},{"key":"e_1_2_1_2_1","unstructured":"2023. IFTTT."},{"key":"e_1_2_1_3_1","unstructured":"2023. Microsoft Power Automate."},{"key":"e_1_2_1_4_1","unstructured":"2023. openHAB."},{"key":"e_1_2_1_5_1","unstructured":"2023. SmartThings."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.3390\/s22030701"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300782"},{"key":"e_1_2_1_8_1","volume-title":"Peter V Desouza, Jennifer C Lai, and Robert L Mercer.","author":"Brown Peter F","year":"1992","unstructured":"Peter F Brown, Vincent J Della Pietra, Peter V Desouza, Jennifer C Lai, and Robert L Mercer. 1992. Class-based n-gram models of natural language. Computational linguistics 18, 4 (1992), 467--480."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979249"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132031"},{"key":"e_1_2_1_11_1","volume-title":"Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289","author":"Clevert Djork-Arn\u00e9","year":"2015","unstructured":"Djork-Arn\u00e9 Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 (2015)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2012.328"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432225"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377325.3377499"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3057859"},{"key":"e_1_2_1_16_1","volume-title":"A semantic blocks model for human activity prediction in smart environments using time-windowed contextual data. Journal of Reliable Intelligent Environments","author":"Dunne Rob","year":"2022","unstructured":"Rob Dunne, Tim Morris, and Simon Harper. 2022. A semantic blocks model for human activity prediction in smart environments using time-windowed contextual data. Journal of Reliable Intelligent Environments (2022), 1--9."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPW.2019.00036"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2902362"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2805830"},{"key":"e_1_2_1_21_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_22_1","unstructured":"Thorin Klosowski. 2016. Automation showdown: IFTTT vs Zapier vs Microsoft Flow."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161201"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510155"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366030.3366123"},{"key":"e_1_2_1_26_1","volume-title":"Towards a natural perspective of smart homes for practical security and safety analyses. In 2020 ieee symposium on security and privacy (sp)","author":"Manandhar Sunil","unstructured":"Sunil Manandhar, Kevin Moran, Kaushal Kafle, Ruhao Tang, Denys Poshyvanyk, and Adwait Nadkarni. 2020. Towards a natural perspective of smart homes for practical security and safety analyses. In 2020 ieee symposium on security and privacy (sp). IEEE, 482--499."},{"key":"e_1_2_1_27_1","volume-title":"Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems 26","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems 26 (2013)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783408"},{"key":"e_1_2_1_29_1","volume-title":"Janardhan Rao Doppa, and Diane J Cook","author":"Minor Bryan David","year":"2017","unstructured":"Bryan David Minor, Janardhan Rao Doppa, and Diane J Cook. 2017. Learning activity predictors from sensor data: Algorithms, evaluation, and applications. IEEE transactions on knowledge and data engineering 29, 12 (2017), 2744--2757."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3197618"},{"key":"e_1_2_1_31_1","volume-title":"Yu Min Hwang, Seok-Kap Ko, and Byung-Tak Lee.","author":"Park Sangjun","year":"2022","unstructured":"Sangjun Park, Hyung Ok Lee, Yu Min Hwang, Seok-Kap Ko, and Byung-Tak Lee. 2022. Enhanced Prediction Model for Human Activity Using an End-to-End Approach. IEEE Internet of Things Journal (2022)."},{"key":"e_1_2_1_32_1","first-page":"4816","article-title":"Mauve: Measuring the gap between neural text and human text using divergence frontiers","volume":"34","author":"Pillutla Krishna","year":"2021","unstructured":"Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances in Neural Information Processing Systems 34 (2021), 4816--4828.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_33_1","volume-title":"Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084","author":"Reimers Nils","year":"2019","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)."},{"key":"e_1_2_1_34_1","volume-title":"A stochastic approximation method. The annals of mathematical statistics","author":"Robbins Herbert","year":"1951","unstructured":"Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method. The annals of mathematical statistics (1951), 400--407."},{"key":"e_1_2_1_35_1","volume-title":"A contrastive framework for neural text generation. arXiv preprint arXiv:2202.06417","author":"Su Yixuan","year":"2022","unstructured":"Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. arXiv preprint arXiv:2202.06417 (2022)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24646-6_10"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/IE.2018.00014"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858556"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409635.1409637"},{"key":"e_1_2_1_40_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_41_1","volume-title":"Predicting Activities of Daily Living for the Coming Time Period in Smart Homes","author":"Wang Wei","year":"2022","unstructured":"Wei Wang, Jianbo Li, Ying Li, and Xueshi Dong. 2022. Predicting Activities of Daily Living for the Coming Time Period in Smart Homes. IEEE Transactions on Human-Machine Systems (2022)."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025617"},{"key":"e_1_2_1_43_1","unstructured":"Imam Nur Bani Yusuf Lingxiao Jiang and David Lo. 2022. Accurate Generation of Trigger-Action Programs with Domain-Adapted Sequence-to-Sequence Learning. (2022)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00043"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432192"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445567"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210080"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631427","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3631427","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,27]],"date-time":"2025-08-27T17:00:12Z","timestamp":1756314012000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631427"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,19]]}},"alternative-id":["10.1145\/3631427"],"URL":"https:\/\/doi.org\/10.1145\/3631427","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,19]]},"assertion":[{"value":"2024-01-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}