{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T16:35:16Z","timestamp":1778344516123,"version":"3.51.4"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,12,30]],"date-time":"2020-12-30T00:00:00Z","timestamp":1609286400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF-CRI Award","award":["1305302"],"award-info":[{"award-number":["1305302"]}]},{"name":"National Science Foundation for funding the research under the Research Experiences for Undergraduates site Programs","award":["CCF-1659807 and CNS-1757893"],"award-info":[{"award-number":["CCF-1659807 and CNS-1757893"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2021,1,31]]},"abstract":"<jats:p>This article presents an in-depth experimental study of Ensemble Deep Learning techniques on small datasets for the analysis of time-series data generated by wearable devices. Deep Learning networks generally require large datasets for training. In some health care applications, such as the real-time smartwatch-based fall detection, there are no publicly available, large, annotated datasets that can be used for training, due to the nature of the problem (i.e., a fall is not a common event). We conducted a series of offline experiments using two different datasets of simulated falls for training various ensemble models. Our offline experimental results show that an ensemble of Recurrent Neural Network (RNN) models, combined by the stacking ensemble technique, outperforms a single RNN model trained on the same data samples. Nonetheless, fall detection models trained on simulated falls and activities of daily living performed by test subjects in a controlled environment, suffer from low precision due to high false-positive rates. In this work, through a set of real-world experiments, we demonstrate that the low precision can be mitigated via the collection of false-positive feedback by the end-users. The final Ensemble RNN model, after re-training with real-world user archived data and feedback, achieved a significantly higher precision without reducing much of the recall in a real-world setting.<\/jats:p>","DOI":"10.1145\/3428666","type":"journal-article","created":{"date-parts":[[2020,12,31]],"date-time":"2020-12-31T05:06:56Z","timestamp":1609391216000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Ensemble Deep Learning on Wearables Using Small Datasets"],"prefix":"10.1145","volume":"2","author":[{"given":"Taylor","family":"Mauldin","sequence":"first","affiliation":[{"name":"Texas State University, San Marcos, Texas"}]},{"given":"Anne H.","family":"Ngu","sequence":"additional","affiliation":[{"name":"Texas State University, San Marcos, Texas"}]},{"given":"Vangelis","family":"Metsis","sequence":"additional","affiliation":[{"name":"Texas State University, San Marcos, Texas"}]},{"given":"Marc E.","family":"Canby","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, Illinois"}]}],"member":"320","published-online":{"date-parts":[[2020,12,30]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI\u201915)","author":"Adib Fadel","unstructured":"Fadel Adib , Hongzi Mao , Zachary Kabelac , Dina Katabi , and Robert C. Miller . 2015. Smart homes that monitor breathing and heart rate . In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI\u201915) . ACM, New York, NY, 837--846. DOI:https:\/\/doi.org\/10.1145\/2702123.2702200 10.1145\/2702123.2702200 Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C. Miller. 2015. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI\u201915). ACM, New York, NY, 837--846. DOI:https:\/\/doi.org\/10.1145\/2702123.2702200"},{"key":"e_1_2_1_2_1","unstructured":"Apple Inc. 2019. Apple Watch Series 4. Retrieved from http:\/\/www.apple.com\/apple-watch-series-4\/activity\/.  Apple Inc. 2019. Apple Watch Series 4. Retrieved from http:\/\/www.apple.com\/apple-watch-series-4\/activity\/."},{"key":"e_1_2_1_3_1","volume-title":"Biomedical signals: Origin and dynamic characteristics","author":"Bronzino Joseph D.","unstructured":"Joseph D. Bronzino . 2006. Biomedical signals: Origin and dynamic characteristics ; frequency-domain analysis. In Medical Devices and Systems. CRC Press , 27--48. Joseph D. Bronzino. 2006. Biomedical signals: Origin and dynamic characteristics; frequency-domain analysis. In Medical Devices and Systems. CRC Press, 27--48."},{"key":"e_1_2_1_4_1","volume-title":"A study on the application of convolutional neural networks to fall detection evaluated with multiple public datasets. Sensors 20, 5","author":"Casilari Eduardo","year":"2020","unstructured":"Eduardo Casilari , Raul Lora-Rivera , and Francisco Garcia-Lagos . 2020. A study on the application of convolutional neural networks to fall detection evaluated with multiple public datasets. Sensors 20, 5 ( 2020 ). DOI:https:\/\/doi.org\/10.3390\/s20051466 10.3390\/s20051466 Eduardo Casilari, Raul Lora-Rivera, and Francisco Garcia-Lagos. 2020. A study on the application of convolutional neural networks to fall detection evaluated with multiple public datasets. Sensors 20, 5 (2020). DOI:https:\/\/doi.org\/10.3390\/s20051466"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1519\/JPT.0b013e3182abe779"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB\u201918)","author":"Chen Xuhui","year":"2018","unstructured":"Xuhui Chen , Jinlong Ji , Tianxi Ji , and Pan Li . 2018 . Cost-sensitive deep active learning for epileptic seizure detection . In Proceedings of the ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB\u201918) . ACM, New York, NY, 226--235. DOI:https:\/\/doi.org\/10.1145\/3233547.3233566 10.1145\/3233547.3233566 Xuhui Chen, Jinlong Ji, Tianxi Ji, and Pan Li. 2018. Cost-sensitive deep active learning for epileptic seizure detection. In Proceedings of the ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB\u201918). ACM, New York, NY, 226--235. DOI:https:\/\/doi.org\/10.1145\/3233547.3233566"},{"key":"e_1_2_1_7_1","volume-title":"Stevens","author":"Cutler Adele","year":"2012","unstructured":"Adele Cutler , D. Richard Cutler , and John R . Stevens . 2012 . Random Forests. Springer , Boston, MA, 157--175. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-9326-7_5 10.1007\/978-1-4419-9326-7_5 Adele Cutler, D. Richard Cutler, and John R. Stevens. 2012. Random Forests. Springer, Boston, MA, 157--175. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-9326-7_5"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-9473(01)00065-2"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.1888"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3090076"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3090076"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the International Conference for Smart Health (ICSH\u201917)","author":"Hee Ngu Anne","year":"2017","unstructured":"Ngu Anne Hee , Wu Yeahuay , Zare Habil , Polican Andrew , Yarbrough Brock , and Yao Lina . 2017 . Fall detection using smartwatchsensor data with accessor architecture . In Proceedings of the International Conference for Smart Health (ICSH\u201917) . Ngu Anne Hee, Wu Yeahuay, Zare Habil, Polican Andrew, Yarbrough Brock, and Yao Lina. 2017. Fall detection using smartwatchsensor data with accessor architecture. In Proceedings of the International Conference for Smart Health (ICSH\u201917)."},{"key":"e_1_2_1_13_1","unstructured":"Andreas Janecek Wilfried Gansterer Michael Demel and Gerhard Ecker. 2008. On the relationship between feature selection and classification accuracy. In New Challenges for Feature Selection in Data Mining and Knowledge Discovery. 90--105.  Andreas Janecek Wilfried Gansterer Michael Demel and Gerhard Ecker. 2008. On the relationship between feature selection and classification accuracy. In New Challenges for Feature Selection in Data Mining and Knowledge Discovery. 90--105."},{"key":"e_1_2_1_14_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A method for stochastic optimization. Retrieved from https:\/\/Arxiv:1412.6980. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. Retrieved from https:\/\/Arxiv:1412.6980."},{"key":"e_1_2_1_15_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. MIT Press , 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. MIT Press, 1097--1105."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2014.09.005"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the IEEE International Conference on Big Data and Smart Computing (BigComp\u201917)","author":"Lee Song-Mi","year":"2017","unstructured":"Song-Mi Lee , Sang Min Yoon , and Heeryon Cho . 2017 . Human activity recognition from accelerometer data using convolutional neural network . In Proceedings of the IEEE International Conference on Big Data and Smart Computing (BigComp\u201917) . IEEE, 131--134. Song-Mi Lee, Sang Min Yoon, and Heeryon Cho. 2017. Human activity recognition from accelerometer data using convolutional neural network. In Proceedings of the IEEE International Conference on Big Data and Smart Computing (BigComp\u201917). IEEE, 131--134."},{"key":"e_1_2_1_18_1","volume-title":"SmartFall: A smartwatch-based fall detection system using deep learning. Sensors 18, 10","author":"Mauldin Taylor R.","year":"2018","unstructured":"Taylor R. Mauldin , Marc E. Canby , Vangelis Metsis , Anne H. H. Ngu , and Coralys Cubero Rivera . 2018. SmartFall: A smartwatch-based fall detection system using deep learning. Sensors 18, 10 ( 2018 ). DOI:https:\/\/doi.org\/10.3390\/s18103363 10.3390\/s18103363 Taylor R. Mauldin, Marc E. Canby, Vangelis Metsis, Anne H. H. Ngu, and Coralys Cubero Rivera. 2018. SmartFall: A smartwatch-based fall detection system using deep learning. Sensors 18, 10 (2018). DOI:https:\/\/doi.org\/10.3390\/s18103363"},{"key":"e_1_2_1_19_1","volume-title":"UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7, 10","author":"Micucci Daniela","year":"2017","unstructured":"Daniela Micucci , Marco Mobilio , and Paolo Napoletano . 2017. UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7, 10 ( 2017 ). DOI:https:\/\/doi.org\/10.3390\/app7101101 10.3390\/app7101101 Daniela Micucci, Marco Mobilio, and Paolo Napoletano. 2017. UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7, 10 (2017). DOI:https:\/\/doi.org\/10.3390\/app7101101"},{"key":"e_1_2_1_20_1","volume-title":"Nicola Blago, Tullio Facchinetti, and Marco Piastra.","author":"Musci Mirto","year":"2018","unstructured":"Mirto Musci , Daniele De Martini , Nicola Blago, Tullio Facchinetti, and Marco Piastra. 2018 . Online fall detection using recurrent neural networks. Retrieved from https:\/\/Arxiv:1804.04976. Mirto Musci, Daniele De Martini, Nicola Blago, Tullio Facchinetti, and Marco Piastra. 2018. Online fall detection using recurrent neural networks. Retrieved from https:\/\/Arxiv:1804.04976."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2018.03.056"},{"key":"e_1_2_1_22_1","unstructured":"RightMinder. 2018. RightMinder\u2014Fall Detection for Android Smartwatches and Android Phones. Retrieved from http:\/\/www.rightminder.com.  RightMinder. 2018. RightMinder\u2014Fall Detection for Android Smartwatches and Android Phones. Retrieved from http:\/\/www.rightminder.com."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2827358"},{"key":"e_1_2_1_24_1","volume-title":"Kayo Henrique de Carvalho Monteiro, Elisson da Silva Rocha, Ivanovitch Silva, and Theo Lynn.","author":"Santos Guto Leoni","year":"2019","unstructured":"Guto Leoni Santos , Patricia Takako Endo , Kayo Henrique de Carvalho Monteiro, Elisson da Silva Rocha, Ivanovitch Silva, and Theo Lynn. 2019 . Accelerometer-based human fall detection using convolutional neural networks. Sensors 19, 7 (2019). DOI:https:\/\/doi.org\/10.3390\/s19071644 10.3390\/s19071644 Guto Leoni Santos, Patricia Takako Endo, Kayo Henrique de Carvalho Monteiro, Elisson da Silva Rocha, Ivanovitch Silva, and Theo Lynn. 2019. Accelerometer-based human fall detection using convolutional neural networks. Sensors 19, 7 (2019). DOI:https:\/\/doi.org\/10.3390\/s19071644"},{"key":"e_1_2_1_25_1","volume-title":"Analysis of a smartphone-based architecture with multiple mobility sensors for fall detection with supervised learning. Sensors 18, 4","author":"Santoyo-Ramon Jose Antonio","year":"2018","unstructured":"Jose Antonio Santoyo-Ramon , Eduardo Casilari , and Jose Manuel Cano-Garcia . 2018. Analysis of a smartphone-based architecture with multiple mobility sensors for fall detection with supervised learning. Sensors 18, 4 ( 2018 ). DOI:https:\/\/doi.org\/10.3390\/s18041155 10.3390\/s18041155 Jose Antonio Santoyo-Ramon, Eduardo Casilari, and Jose Manuel Cano-Garcia. 2018. Analysis of a smartphone-based architecture with multiple mobility sensors for fall detection with supervised learning. Sensors 18, 4 (2018). DOI:https:\/\/doi.org\/10.3390\/s18041155"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS\u201916)","author":"Shin S.","year":"2016","unstructured":"S. Shin and W. Sung . 2016. Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks . In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS\u201916) . 2274--2277. DOI:https:\/\/doi.org\/10.1109\/ISCAS. 2016 .7539037 10.1109\/ISCAS.2016.7539037 S. Shin and W. Sung. 2016. Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS\u201916). 2274--2277. DOI:https:\/\/doi.org\/10.1109\/ISCAS.2016.7539037"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.3390\/s17010198"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops. 258--261","author":"Tacconi C.","year":"2011","unstructured":"C. Tacconi , S. Mellone , and L. Chiari . 2011. Smartphone-based applications for investigating falls and mobility . In Proceedings of the 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops. 258--261 . DOI:https:\/\/doi.org\/10.4108\/icst.pervasivehealth. 2011 .246060 10.4108\/icst.pervasivehealth.2011.246060 C. Tacconi, S. Mellone, and L. Chiari. 2011. Smartphone-based applications for investigating falls and mobility. In Proceedings of the 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops. 258--261. DOI:https:\/\/doi.org\/10.4108\/icst.pervasivehealth.2011.246060"},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"T. Theodoridis V. Solachidis N. Vretos and P. Daras. 2018. Human fall detection from acceleration measurements using a recurrent neural network. In Precision Medicine Powered by pHealth and Connected Health. Springer 145--149.  T. Theodoridis V. Solachidis N. Vretos and P. Daras. 2018. Human fall detection from acceleration measurements using a recurrent neural network. In Precision Medicine Powered by pHealth and Connected Health. Springer 145--149.","DOI":"10.1007\/978-981-10-7419-6_25"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2018.02.010"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the International Symposium in Sensing and Instrumentation in IoT Era (ISSI\u201918)","author":"Yu S.","year":"2018","unstructured":"S. Yu , L. Qin , and Q. Yin . 2018. A C-LSTM neural network for human activity recognition using wearables . In Proceedings of the International Symposium in Sensing and Instrumentation in IoT Era (ISSI\u201918) . 1--6. DOI:https:\/\/doi.org\/10.1109\/ISSI. 2018 .8538129 10.1109\/ISSI.2018.8538129 S. Yu, L. Qin, and Q. Yin. 2018. A C-LSTM neural network for human activity recognition using wearables. In Proceedings of the International Symposium in Sensing and Instrumentation in IoT Era (ISSI\u201918). 1--6. DOI:https:\/\/doi.org\/10.1109\/ISSI.2018.8538129"},{"key":"e_1_2_1_32_1","volume-title":"Ensemble Machine Learning: Methods and Applications","author":"Zhang Cha","unstructured":"Cha Zhang and Yunqian Ma. 2012. Ensemble Machine Learning: Methods and Applications . Springer . Cha Zhang and Yunqian Ma. 2012. Ensemble Machine Learning: Methods and Applications. Springer."},{"key":"e_1_2_1_33_1","volume-title":"2017 36th Chinese Control Conference (CCC'17)","author":"Zhiqiang Wang","year":"2017","unstructured":"Wang Zhiqiang and Liu Jun . 2017 . A review of object detection based on convolutional neural network . In 2017 36th Chinese Control Conference (CCC'17) . IEEE, 11104--11109. Wang Zhiqiang and Liu Jun. 2017. A review of object detection based on convolutional neural network. In 2017 36th Chinese Control Conference (CCC'17). IEEE, 11104--11109."},{"key":"e_1_2_1_34_1","volume-title":"An analysis on sensor locations of the human body for wearable fall detection devices: Principles and practice. Sensors 16, 8","author":"Ozdemir Ahmet Turan","year":"2016","unstructured":"Ahmet Turan Ozdemir . 2016. An analysis on sensor locations of the human body for wearable fall detection devices: Principles and practice. Sensors 16, 8 ( 2016 ). DOI:https:\/\/doi.org\/10.3390\/s16081161 10.3390\/s16081161 Ahmet Turan Ozdemir. 2016. An analysis on sensor locations of the human body for wearable fall detection devices: Principles and practice. Sensors 16, 8 (2016). DOI:https:\/\/doi.org\/10.3390\/s16081161"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3428666","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3428666","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:01:59Z","timestamp":1750197719000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3428666"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,30]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1,31]]}},"alternative-id":["10.1145\/3428666"],"URL":"https:\/\/doi.org\/10.1145\/3428666","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"value":"2691-1957","type":"print"},{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,30]]},"assertion":[{"value":"2019-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-12-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}