{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T07:37:28Z","timestamp":1767339448933,"version":"3.41.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T00:00:00Z","timestamp":1624233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2021,8,31]]},"abstract":"<jats:p>Forecasting human poses given a sequence of historical pose frames has several important applications, especially in the domain of smart home safety. Recently, computer vision-based human pose forecasting has made a breakthrough using deep learning technology. However, to implement a practical system deployed on an IoT edge environment, there are still two issues to be addressed. First, existing methods on pose forecasting fail to model the coherent structural information of connected human joints and thus cannot achieve satisfactory prediction accuracy, especially for long-term predictions. Second, a general and static pre-trained prediction model may not perform well in the deployment environment due to the visual domain shift problem. In this article, we propose a hybrid cloud-edge system called GPFS to solve those issues. Specifically, we first introduce a novel graph convolutional neural network (GCN)-based sequence-to-sequence learning method, which enhances the sequence encoder by using a graph to represent both the spatial and temporal connections of the human joints in the input frames. The GCN improves the forecasting accuracy by capturing the motion pattern of each joint as well as the correlations among different human joints over time. Second, to address the domain shift issue and protect data privacy, we extend the system to perform online learning on the IoT edge to adapt the cloud trained general model with online collected on-site domain data. Extensive evaluation on Human 3.6M and Penn Action datasets demonstrates the superiority of our proposed system.<\/jats:p>","DOI":"10.1145\/3460199","type":"journal-article","created":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T20:19:31Z","timestamp":1624306771000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["GPFS: A Graph-based Human Pose Forecasting System for Smart Home with Online Learning"],"prefix":"10.1145","volume":"17","author":[{"given":"Xin","family":"Li","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dawei","family":"Li","sequence":"additional","affiliation":[{"name":"Samsung Research America, Mountain View, CA, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,6,21]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00560"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Z. Cao G. Hidalgo T. Simon S.-E. Wei and Y. Sheikh. 2019. \u201cOpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields.\u201d IEEE transactions on pattern analysis and machine intelligence 43 1 (2019) 172\u2013186.  Z. Cao G. Hidalgo T. Simon S.-E. Wei and Y. Sheikh. 2019. \u201cOpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields.\u201d IEEE transactions on pattern analysis and machine intelligence 43 1 (2019) 172\u2013186.","DOI":"10.1109\/TPAMI.2019.2929257"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.388"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2019.102897"},{"key":"e_1_2_1_5_1","volume-title":"2019 IEEE Winter Conference on Applications of Computer Vision (WACV\u201919)","author":"Adeli Ehsan","year":"2019","unstructured":"Hsu-kuang Chiu, Ehsan Adeli , Borui Wang , De-An Huang , and Juan Carlos Niebles . 2019 . Action-agnostic human pose forecasting . In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV\u201919) . IEEE, 1423\u20131432. Hsu-kuang Chiu, Ehsan Adeli, Borui Wang, De-An Huang, and Juan Carlos Niebles. 2019. Action-agnostic human pose forecasting. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV\u201919). IEEE, 1423\u20131432."},{"key":"e_1_2_1_6_1","volume-title":"Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. arXiv preprint arXiv:1802.07007","author":"Cui Zhiyong","year":"2018","unstructured":"Zhiyong Cui , Kristian Henrickson , Ruimin Ke , and Yinhai Wang . 2018. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. arXiv preprint arXiv:1802.07007 ( 2018 ). Zhiyong Cui, Kristian Henrickson, Ruimin Ke, and Yinhai Wang. 2018. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. arXiv preprint arXiv:1802.07007 (2018)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313488"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/2919332.2919834"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/3045118.3045244"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305510"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2017.00059"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.248"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.573"},{"key":"e_1_2_1_14_1","volume-title":"Kipf and Max Welling","author":"Thomas","year":"2016","unstructured":"Thomas N. Kipf and Max Welling . 2016 . Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016). Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2430335"},{"volume-title":"2016 IEEE\/ACM Symposium on Edge Computing (SEC\u201916)","author":"Li D.","key":"e_1_2_1_16_1","unstructured":"D. Li , T. Salonidis , N. V. Desai , and M. C. Chuah . 2016. DeepCham: Collaborative edge-mediated adaptive deep learning for mobile object recognition . In 2016 IEEE\/ACM Symposium on Edge Computing (SEC\u201916) . 64\u201376. D. Li, T. Salonidis, N. V. Desai, and M. C. Chuah. 2016. DeepCham: Collaborative edge-mediated adaptive deep learning for mobile object recognition. In 2016 IEEE\/ACM Symposium on Edge Computing (SEC\u201916). 64\u201376."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318216.3363317"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2773081"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00587"},{"volume-title":"Proceedings of the IEEE International Conference on Computer Vision. 1744\u20131752","author":"Liang Xiaodan","key":"e_1_2_1_20_1","unstructured":"Xiaodan Liang , Lisa Lee , Wei Dai , and Eric P. Xing . 2017. Dual motion GAN for future-flow embedded video prediction . In Proceedings of the IEEE International Conference on Computer Vision. 1744\u20131752 . Xiaodan Liang, Lisa Lee, Wei Dai, and Eric P. Xing. 2017. Dual motion GAN for future-flow embedded video prediction. In Proceedings of the IEEE International Conference on Computer Vision. 1744\u20131752."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.478"},{"key":"e_1_2_1_22_1","volume-title":"Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104","author":"Lotter William","year":"2016","unstructured":"William Lotter , Gabriel Kreiman , and David Cox . 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 ( 2016 ). William Lotter, Gabriel Kreiman, and David Cox. 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 (2016)."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.497"},{"key":"e_1_2_1_24_1","volume-title":"Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440","author":"Mathieu Michael","year":"2015","unstructured":"Michael Mathieu , Camille Couprie , and Yann LeCun . 2015. Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440 ( 2015 ). Michael Mathieu, Camille Couprie, and Yann LeCun. 2015. Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440 (2015)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392410"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107561"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969442.2969560"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3455008"},{"key":"e_1_2_1_29_1","volume-title":"Spatio-temporal video autoencoder with differentiable memory. arXiv preprint arXiv:1511.06309","author":"Patraucean Viorica","year":"2015","unstructured":"Viorica Patraucean , Ankur Handa , and Roberto Cipolla . 2015. Spatio-temporal video autoencoder with differentiable memory. arXiv preprint arXiv:1511.06309 ( 2015 ). Viorica Patraucean, Ankur Handa, and Roberto Cipolla. 2015. Spatio-temporal video autoencoder with differentiable memory. arXiv preprint arXiv:1511.06309 (2015)."},{"key":"e_1_2_1_30_1","volume-title":"Quaternet: A quaternion-based recurrent model for human motion. arXiv preprint arXiv:1805.06485","author":"Pavllo Dario","year":"2018","unstructured":"Dario Pavllo , David Grangier , and Michael Auli . 2018 . Quaternet: A quaternion-based recurrent model for human motion. arXiv preprint arXiv:1805.06485 (2018). Dario Pavllo, David Grangier, and Michael Auli. 2018. Quaternet: A quaternion-based recurrent model for human motion. arXiv preprint arXiv:1805.06485 (2018)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00247"},{"key":"e_1_2_1_32_1","volume-title":"Webb","author":"Pervin Edward","year":"1982","unstructured":"Edward Pervin and Jon A . Webb . 1982 . Quaternions in Computer Vision and Robotics. Technical Report. Carnegie-Mellon UNIV Pittsburgh PA Dept of Computer Science . Edward Pervin and Jon A. Webb. 1982. Quaternions in Computer Vision and Robotics. Technical Report. Carnegie-Mellon UNIV Pittsburgh PA Dept of Computer Science."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.132"},{"key":"e_1_2_1_34_1","volume-title":"Next-flow: Hybrid multi-tasking with next-frame prediction to boost optical-flow estimation in the wild. arXiv preprint arXiv:1612.03777 1, 2","author":"Sedaghat Nima","year":"2016","unstructured":"Nima Sedaghat . 2016 . Next-flow: Hybrid multi-tasking with next-frame prediction to boost optical-flow estimation in the wild. arXiv preprint arXiv:1612.03777 1, 2 (2016), 6. Nima Sedaghat. 2016. Next-flow: Hybrid multi-tasking with next-frame prediction to boost optical-flow estimation in the wild. arXiv preprint arXiv:1612.03777 1, 2 (2016), 6."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01230"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/3045118.3045209"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157096.3157107"},{"key":"e_1_2_1_40_1","volume-title":"32nd AAAI Conference on Artificial Intelligence.","author":"Yan Sijie","year":"2018","unstructured":"Sijie Yan , Yuanjun Xiong , and Dahua Lin . 2018 . Spatial temporal graph convolutional networks for skeleton-based action recognition . In 32nd AAAI Conference on Artificial Intelligence. Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In 32nd AAAI Conference on Artificial Intelligence."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01249-6_13"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/2586117.2587158"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460199","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3460199","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:19Z","timestamp":1750188619000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460199"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,21]]},"references-count":42,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,8,31]]}},"alternative-id":["10.1145\/3460199"],"URL":"https:\/\/doi.org\/10.1145\/3460199","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"type":"print","value":"1550-4859"},{"type":"electronic","value":"1550-4867"}],"subject":[],"published":{"date-parts":[[2021,6,21]]},"assertion":[{"value":"2020-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}