{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T20:18:04Z","timestamp":1772828284527,"version":"3.50.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,3,18]],"date-time":"2020-03-18T00:00:00Z","timestamp":1584489600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2020,3,18]]},"abstract":"<jats:p>The paper enhances deep-neural-network-based inference in sensing applications by introducing a lightweight attention mechanism called the global attention module for multi-sensor information fusion. This mechanism is capable of utilizing information collected from higher layers of the neural network to selectively amplify the influence of informative features and suppress unrelated noise at the fusion layer. We successfully integrate this mechanism into a new end-to-end learning framework, called GIobalFusion, where two global attention modules are deployed for spatial fusion and sensing modality fusion, respectively. Through an extensive evaluation on four public human activity recognition (HAR) datasets, we successfully demonstrate the effectiveness of GlobalFusion at improving information fusion quality. The new approach outperforms the state-of-the-art algorithms on all four datasets with a clear margin. We also show that the learned attention weights agree well with human intuition. We then validate the efficiency of GlobalFusion by testing its inference time and energy consumption on commodity IoT devices. Only a negligible overhead is induced by the global attention modules.<\/jats:p>","DOI":"10.1145\/3380999","type":"journal-article","created":{"date-parts":[[2020,3,18]],"date-time":"2020-03-18T18:54:31Z","timestamp":1584557671000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":51,"title":["GIobalFusion"],"prefix":"10.1145","volume":"4","author":[{"given":"Shengzhong","family":"Liu","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuochao","family":"Yao","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinyang","family":"Li","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongxin","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianshi","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huajie","family":"Shao","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tarek","family":"Abdelzaher","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,3,18]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n.d.]. Monsoon High Voltage Power Monitor. https:\/\/www.msoon.com\/online-store\/High-Voltage-Power-Monitor-Part-Number-AAA10F-p90002590  [n.d.]. Monsoon High Voltage Power Monitor. https:\/\/www.msoon.com\/online-store\/High-Voltage-Power-Monitor-Part-Number-AAA10F-p90002590"},{"key":"e_1_2_1_2_1","unstructured":"[n.d.]. TensorFlow Lite Interpreter. https:\/\/www.tensorflow.org\/lite\/guide\/inference  [n.d.]. TensorFlow Lite Interpreter. https:\/\/www.tensorflow.org\/lite\/guide\/inference"},{"key":"e_1_2_1_3_1","volume-title":"Tensorflow: A system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16). 265--283.","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , 2016 . Tensorflow: A system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16). 265--283. Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16). 265--283."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2010.04.019"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2009.14"},{"key":"e_1_2_1_6_1","volume-title":"Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473","author":"Bahdanau Dzmitry","year":"2014","unstructured":"Dzmitry Bahdanau , Kyunghyun Cho , and Yoshua Bengio . 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 ( 2014 ). Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994551.2994564"},{"key":"e_1_2_1_8_1","volume-title":"An attentive survey of attention models. arXiv preprint arXiv:1904.02874","author":"Chaudhari Sneha","year":"2019","unstructured":"Sneha Chaudhari , Gungor Polatkan , Rohan Ramanath , and Varun Mithal . 2019. An attentive survey of attention models. arXiv preprint arXiv:1904.02874 ( 2019 ). Sneha Chaudhari, Gungor Polatkan, Rohan Ramanath, and Varun Mithal. 2019. An attentive survey of attention models. arXiv preprint arXiv:1904.02874 (2019)."},{"key":"e_1_2_1_9_1","unstructured":"Yunpeng Chen Yannis Kalantidis Jianshu Li Shuicheng Yan and Jiashi Feng. 2018. A 2-Nets: Double Attention Networks. In Advances in Neural Information Processing Systems. 352--361.  Yunpeng Chen Yannis Kalantidis Jianshu Li Shuicheng Yan and Jiashi Feng. 2018. A 2-Nets: Double Attention Networks. In Advances in Neural Information Processing Systems. 352--361."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_1_11_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_2_1_12_1","volume-title":"Multisensor data fusion","author":"Hall David","unstructured":"David Hall and James Llinas . 2001. Multisensor data fusion . CRC press . David Hall and James Llinas. 2001. Multisensor data fusion. CRC press."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00745"},{"key":"e_1_2_1_15_1","volume-title":"Learn to pay attention. arXiv preprint arXiv:1804.02391","author":"Jetley Saumya","year":"2018","unstructured":"Saumya Jetley , Nicholas A Lord , Namhoon Lee , and Philip HS Torr . 2018. Learn to pay attention. arXiv preprint arXiv:1804.02391 ( 2018 ). Saumya Jetley, Nicholas A Lord, Namhoon Lee, and Philip HS Torr. 2018. Learn to pay attention. arXiv preprint arXiv:1804.02391 (2018)."},{"key":"e_1_2_1_16_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_17_1","unstructured":"Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.  Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2959355.2959378"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2017.2940968"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3356250.3360041"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307334.3326109"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2017.7953301"},{"key":"e_1_2_1_23_1","volume-title":"Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025","author":"Luong Minh-Thang","year":"2015","unstructured":"Minh-Thang Luong , Hieu Pham , and Christopher D Manning . 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 ( 2015 ). Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2461544"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2968219.2971461"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161174"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2012.13"},{"key":"e_1_2_1_28_1","unstructured":"Oren Rippel Jasper Snoek and Ryan P Adams. 2015. Spectral representations for convolutional neural networks. In Advances in neural information processing systems. 2449--2457.  Oren Rippel Jasper Snoek and Ryan P Adams. 2015. Spectral representations for convolutional neural networks. In Advances in neural information processing systems. 2449--2457."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3055031.3055087"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2906388.2906407"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOM.2016.7456521"},{"key":"e_1_2_1_32_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_2_1_33_1","volume-title":"Multi-Level Sensor Fusion with Deep Learning. CoRR abs\/1811.02447","author":"Vielzeuf Valentin","year":"2018","unstructured":"Valentin Vielzeuf , Alexis Lechervy , St\u00e9phane Pateux , and Fr\u00e9d\u00e9ric Jurie . 2018. Multi-Level Sensor Fusion with Deep Learning. CoRR abs\/1811.02447 ( 2018 ). arXiv:1811.02447 http:\/\/arxiv.org\/abs\/1811.02447 Valentin Vielzeuf, Alexis Lechervy, St\u00e9phane Pateux, and Fr\u00e9d\u00e9ric Jurie. 2018. Multi-Level Sensor Fusion with Deep Learning. CoRR abs\/1811.02447 (2018). arXiv:1811.02447 http:\/\/arxiv.org\/abs\/1811.02447"},{"key":"e_1_2_1_34_1","volume-title":"Amanda C De C Williams, and Nadia Bianchi-Berthouze","author":"Wang Chongyang","year":"2019","unstructured":"Chongyang Wang , Min Peng , Temitayo A Olugbade , Nicholas D Lane , Amanda C De C Williams, and Nadia Bianchi-Berthouze . 2019 . Learning Bodily and Temporal Attention in Protective Movement Behavior Detection . arXiv preprint arXiv:1904.10824 (2019). Chongyang Wang, Min Peng, Temitayo A Olugbade, Nicholas D Lane, Amanda C De C Williams, and Nadia Bianchi-Berthouze. 2019. Learning Bodily and Temporal Attention in Protective Movement Behavior Detection. arXiv preprint arXiv:1904.10824 (2019)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00813"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2018.1701277"},{"key":"e_1_2_1_37_1","volume-title":"International conference on machine learning. 2048--2057","author":"Xu Kelvin","year":"2015","unstructured":"Kelvin Xu , Jimmy Ba , Ryan Kiros , Kyunghyun Cho , Aaron Courville , Ruslan Salakhudinov , Rich Zemel , and Yoshua Bengio . 2015 . Show, attend and tell: Neural image caption generation with visual attention . In International conference on machine learning. 2048--2057 . Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning. 2048--2057."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPSN.2016.7460721"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3323679.3326513"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052577"},{"key":"e_1_2_1_41_1","volume-title":"Stfnets: Learning sensing signals from the time-frequency perspective with short-time fourier neural networks.","author":"Yao Shuochao","year":"2019","unstructured":"Shuochao Yao , Ailing Piao , Wenjun Jiang , Yiran Zhao , Huajie Shao , Shengzhong Liu , Dongxin Liu , Jinyang Li , Tianshi Wang , Shaohan Hu , 2019 . Stfnets: Learning sensing signals from the time-frequency perspective with short-time fourier neural networks. (2019), 2192--2202. Shuochao Yao, Ailing Piao, Wenjun Jiang, Yiran Zhao, Huajie Shao, Shengzhong Liu, Dongxin Liu, Jinyang Li, Tianshi Wang, Shaohan Hu, et al. 2019. Stfnets: Learning sensing signals from the time-frequency perspective with short-time fourier neural networks. (2019), 2192--2202."},{"key":"e_1_2_1_42_1","volume-title":"SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications","author":"Yao Shuochao","year":"2019","unstructured":"Shuochao Yao , Yiran Zhao , Huajie Shao , Dongxin Liu , Shengzhong Liu , Yifan Hao , Ailing Piao , Shaohan Hu , Su Lu , and Tarek F Abdelzaher . 2019. SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications . In IEEE INFOCOM 2019 -IEEE Conference on Computer Communications. IEEE , 1243--1251. Shuochao Yao, Yiran Zhao, Huajie Shao, Dongxin Liu, Shengzhong Liu, Yifan Hao, Ailing Piao, Shaohan Hu, Su Lu, and Tarek F Abdelzaher. 2019. SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 1243--1251."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2018.2381131"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3131672.3131675"},{"key":"e_1_2_1_45_1","unstructured":"Jason Yosinski Jeff Clune Yoshua Bengio and Hod Lipson. 2014. How transferable are features in deep neural networks?. In Advances in neural information processing systems. 3320--3328.  Jason Yosinski Jeff Clune Yoshua Bengio and Hod Lipson. 2014. How transferable are features in deep neural networks?. In Advances in neural information processing systems. 3320--3328."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3107411.3107419"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267242.3267286"},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Yiran Zhao Shuochao Yao Dongxin Liu Huajie Shao and Shengzhong Liu. 2019. GreenRoute: A Generalizable Fuel-Saving Vehicular Navigation Service. (2019) 1--10.  Yiran Zhao Shuochao Yao Dongxin Liu Huajie Shao and Shengzhong Liu. 2019. GreenRoute: A Generalizable Fuel-Saving Vehicular Navigation Service. (2019) 1--10.","DOI":"10.1109\/ICAC.2019.00011"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3380999","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3380999","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:32:46Z","timestamp":1750199566000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3380999"}},"subtitle":["A Global Attentional Deep Learning Framework for Multisensor Information Fusion"],"short-title":[],"issued":{"date-parts":[[2020,3,18]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,3,18]]}},"alternative-id":["10.1145\/3380999"],"URL":"https:\/\/doi.org\/10.1145\/3380999","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,18]]},"assertion":[{"value":"2020-03-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}