{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T03:47:25Z","timestamp":1766720845593,"version":"3.37.3"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2020,6,27]],"date-time":"2020-06-27T00:00:00Z","timestamp":1593216000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,6,27]],"date-time":"2020-06-27T00:00:00Z","timestamp":1593216000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/T51729X\/1"],"award-info":[{"award-number":["EP\/T51729X\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Pers Ubiquit Comput"],"published-print":{"date-parts":[[2020,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In recent years, machine learning has developed rapidly, enabling the development of applications with high levels of recognition accuracy relating to the use of speech and images. However, other types of data to which these models can be applied have not yet been explored as thoroughly. Labelling is an indispensable stage of data pre-processing that can be particularly challenging, especially when applied to single or multi-model real-time sensor data collection approaches. Currently, real-time sensor data labelling is an unwieldy process, with a limited range of tools available and poor performance characteristics, which can lead to the performance of the machine learning models being compromised. In this paper, we introduce new techniques for labelling at the point of collection coupled with a pilot study and a systematic performance comparison of two popular types of deep neural networks running on five custom built devices and a comparative mobile app (68.5\u201389% accuracy within-device GRU model, 92.8% highest LSTM model accuracy). These devices are designed to enable real-time labelling with various buttons, slide potentiometer and force sensors. This exploratory work illustrates several key features that inform the design of data collection tools that can help researchers select and apply appropriate labelling techniques to their work. We also identify common bottlenecks in each architecture and provide field tested guidelines to assist in building adaptive, high-performance edge solutions.<\/jats:p>","DOI":"10.1007\/s00779-020-01427-x","type":"journal-article","created":{"date-parts":[[2020,6,27]],"date-time":"2020-06-27T18:03:33Z","timestamp":1593281013000},"page":"709-722","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["LabelSens: enabling real-time sensor data labelling at the point of collection using an artificial intelligence-based approach"],"prefix":"10.1007","volume":"24","author":[{"given":"Kieran","family":"Woodward","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1720-0661","authenticated-orcid":false,"given":"Eiman","family":"Kanjo","sequence":"additional","affiliation":[]},{"given":"Andreas","family":"Oikonomou","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2122-8077","authenticated-orcid":false,"given":"Alan","family":"Chamberlain","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,6,27]]},"reference":[{"key":"1427_CR1","unstructured":"Nvidia (2019) NVIDIA Jetson Nano Developer Kit | NVIDIA Developer. https:\/\/developer.nvidia.com\/embedded\/buy\/jetson-nano-devkit."},{"key":"1427_CR2","unstructured":"Google (2019) Google Coral. https:\/\/coral.withgoogle.com\/"},{"key":"1427_CR3","doi-asserted-by":"publisher","first-page":"1192","DOI":"10.1109\/SURV.2012.110112.00192","volume":"15","author":"\u00d3D Lara","year":"2013","unstructured":"Lara \u00d3D, Labrador MA (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutorials 15:1192\u20131209. https:\/\/doi.org\/10.1109\/SURV.2012.110112.00192","journal-title":"IEEE Commun Surv Tutorials"},{"key":"1427_CR4","doi-asserted-by":"crossref","unstructured":"Younis EMG, Kanjo E, Chamberlain A (2019) Designing and evaluating mobile self-reporting techniques: crowdsourcing for citizen science. Pers Ubiquitous Comput:1\u201310","DOI":"10.1007\/s00779-019-01207-2"},{"key":"1427_CR5","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1080\/10508422.2017.1285237","volume":"27","author":"V Kwan","year":"2017","unstructured":"Kwan V, Hagen G, Noel M, Dobson K, Yeates K (2017) Healthcare at your fingertips: the professional ethics of smartphone health-monitoring applications. Ethics Behav 27:615\u2013631. https:\/\/doi.org\/10.1080\/10508422.2017.1285237","journal-title":"Ethics Behav"},{"key":"1427_CR6","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.inffus.2017.05.005","volume":"40","author":"E Kanjo","year":"2018","unstructured":"Kanjo E, Younis EMG, Sherkat N (2018) Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach. Inf Fusion 40:18\u201331","journal-title":"Inf Fusion"},{"key":"1427_CR7","doi-asserted-by":"publisher","first-page":"e0183890","DOI":"10.1371\/journal.pone.0183890","volume":"12","author":"L Al-barrak","year":"2017","unstructured":"Al-barrak L, Kanjo E, Younis EMG (2017) NeuroPlace: categorizing urban places according to mental states. PLoS One 12:e0183890","journal-title":"PLoS One"},{"key":"1427_CR8","doi-asserted-by":"publisher","first-page":"562","DOI":"10.1007\/s11036-009-0217-y","volume":"15","author":"E Kanjo","year":"2010","unstructured":"Kanjo E (2010) NoiseSPY: a real-time mobile phone platform for urban noise monitoring and mapping. Mob Networks Appl 15:562\u2013574. https:\/\/doi.org\/10.1007\/s11036-009-0217-y","journal-title":"Mob Networks Appl"},{"key":"1427_CR9","doi-asserted-by":"publisher","first-page":"22023","DOI":"10.1109\/ACCESS.2017.2755661","volume":"5","author":"E Kanjo","year":"2017","unstructured":"Kanjo E, Kuss DJ, Ang CS (2017) NotiMind: utilizing responses to smart phone notifications as affective sensors. IEEE Access 5:22023\u201322035. https:\/\/doi.org\/10.1109\/ACCESS.2017.2755661","journal-title":"IEEE Access"},{"key":"1427_CR10","unstructured":"Google (2019) reCAPTCHA: Easy on Humans, Hard on Bots. https:\/\/www.google.com\/recaptcha\/intro\/v3.html. Accessed 8 Apr 2019"},{"key":"1427_CR11","unstructured":"Vaughan JW (2019) Making better use of the crowd: how crowdsourcing can advance machine learning research. JMLR 18"},{"key":"1427_CR12","doi-asserted-by":"crossref","unstructured":"Lasecki WS, Song YC, Kautz H, Bigham JP (2012) Real-time crowd labeling for deployable activity recognition","DOI":"10.1145\/2441776.2441912"},{"key":"1427_CR13","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1147\/sj.393.0915","volume":"39","author":"B Ullmer","year":"2000","unstructured":"Ullmer B, Ishii H (2000) Emerging frameworks for tangible user interfaces. IBM Syst J 39:915\u2013931. https:\/\/doi.org\/10.1147\/sj.393.0915","journal-title":"IBM Syst J"},{"key":"1427_CR14","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2007.4373774","volume-title":"Real-time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor","author":"EM Tapia","year":"2007","unstructured":"Tapia EM, Intille SS, Haskell W, Larson K, Wright J, King A, Friedman R (2007) Real-time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor. Proceedings - International Symposium on Wearable Computers, ISWC, In"},{"key":"1427_CR15","doi-asserted-by":"crossref","unstructured":"Kwapisz JR, Weiss GM, Moore SA (2010) Activity recognition using cell phone accelerometers","DOI":"10.1145\/1964897.1964918"},{"key":"1427_CR16","unstructured":"Settles B (2010) Active learning literature survey. Univ Wisconsin, Madison. 10.1.1.167.4245"},{"key":"1427_CR17","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2006.286336","volume-title":"Towards less supervision in activity recognition from wearable sensors","author":"T Hu\u1ef3nh","year":"2006","unstructured":"Hu\u1ef3nh T, Schiele B (2006) Towards less supervision in activity recognition from wearable sensors. Proceedings - International Symposium on Wearable Computers, ISWC, In"},{"key":"1427_CR18","first-page":"510","volume-title":"Hollywood in homes: crowdsourcing data collection for activity understanding","author":"GA Sigurdsson","year":"2016","unstructured":"Sigurdsson GA, Varol G, Wang X, Farhadi A, Laptev I, Gupta A (2016) Hollywood in homes: crowdsourcing data collection for activity understanding. Springer, Cham, pp 510\u2013526"},{"key":"1427_CR19","doi-asserted-by":"crossref","unstructured":"Tsikrika T, Diou C, de Vries AP, Delopoulos A (2009) Image annotation using clickthrough data","DOI":"10.1145\/1646396.1646415"},{"key":"1427_CR20","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1007\/s11263-007-0090-8","volume":"77","author":"BC Russell","year":"2008","unstructured":"Russell BC, Torralba A, Murphy KP, Freeman WT (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77:157\u2013173. https:\/\/doi.org\/10.1007\/s11263-007-0090-8","journal-title":"Int J Comput Vis"},{"key":"1427_CR21","unstructured":"Oramas S, Nieto O, Barbieri F, Serra X (2017) Multi-label music genre classification from audio, Text, and Images Using Deep Features"},{"key":"1427_CR22","unstructured":"Kim YE, Schmidt E, Emelle L (2008) MoodSwings: a collaborative game for music mood label collection. In: ISMIR 2008 - 9th International Conference on Music Information Retrieval"},{"key":"1427_CR23","unstructured":"Law ELM, Ahn L Von, Dannenberg RB, Crawford M (2007) Tagatune: a game for music and sound annotation. In: Proceedings of the 8th International Conference on Music Information Retrieval, ISMIR 2007"},{"key":"1427_CR24","unstructured":"Davidov D, Tsur O, Rappoport A (2010) Enhanced sentiment learning using Twitter Hashtags and smileys"},{"key":"1427_CR25","unstructured":"Liu K-L, Li W-J, Guo M (2012) Emoticon smoothed language models for twitter sentiment analysis. Twenty-Sixth AAAI Conf Artif Intell"},{"key":"1427_CR26","doi-asserted-by":"publisher","unstructured":"Bravo J, Herv\u00e1s R, Villarreal V (2015) Ambient intelligence for health first international conference, AmIHEALTH 2015 Puerto Varas, Chile, December 1\u20134, 2015 proceedings. Lect notes Comput Sci (including Subser Lect notes Artif Intell Lect notes bioinformatics) 9456:189\u2013200 . https:\/\/doi.org\/10.1007\/978-3-319-26508-7","DOI":"10.1007\/978-3-319-26508-7"},{"key":"1427_CR27","first-page":"518","volume-title":"Engaging users in self-reporting their data: a tangible Interface for quantified self","author":"F Sarzotti","year":"2015","unstructured":"Sarzotti F, Lombardi I, Rapp A, Marcengo A, Cena F (2015) Engaging users in self-reporting their data: a tangible Interface for quantified self. Springer, Cham, pp 518\u2013527"},{"key":"1427_CR28","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3390\/computers7010007","volume":"7","author":"F Sarzotti","year":"2018","unstructured":"Sarzotti F (2018) Self-monitoring of emotions and mood using a tangible approach. Computers 7:7","journal-title":"Computers"},{"key":"1427_CR29","unstructured":"Tai Y, Chan C, Hsu JY (2010) Automatic road anomaly detection using smart mobile device. 2010 15th Conf Artif Intell Appl"},{"key":"1427_CR30","doi-asserted-by":"crossref","unstructured":"Kanjo E, Younis EMG, Ang CS (2018) Deep learning analysis of Mobile physiological, Environmental and Location Sensor Data for Emotion Detection J Inf Fusion 1\u201333","DOI":"10.1016\/j.inffus.2018.09.001"},{"key":"1427_CR31","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.inffus.2017.05.005","volume":"40","author":"E Kanjo","year":"2018","unstructured":"Kanjo E, Younis EMG, Sherkat N (2018) Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach. Inf Fusion 40:18\u201331. https:\/\/doi.org\/10.1016\/j.inffus.2017.05.005","journal-title":"Inf Fusion"},{"key":"1427_CR32","unstructured":"Fujitsu (2019) Fujitsu Develops Automatic Labeling Technology to Accelerate AI Use of Time-Series Data - Fujitsu Global. https:\/\/www.fujitsu.com\/global\/about\/resources\/news\/press-releases\/2019\/0510-01.html."},{"key":"1427_CR33","unstructured":"Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks"},{"key":"1427_CR34","doi-asserted-by":"crossref","unstructured":"Pearlmutter (1989) Learning state space trajectories in recurrent neural networks. In: International Joint Conference on Neural Networks. IEEE, pp 365\u2013372 vol.2","DOI":"10.1109\/IJCNN.1989.118724"},{"key":"1427_CR35","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735\u20131780. https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735","journal-title":"Neural Comput"},{"key":"1427_CR36","unstructured":"Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling"},{"key":"1427_CR37","unstructured":"Jozefowicz R, Zaremba W, Sutskever I (2015) An empirical exploration of recurrent network architectures. Proc. 32nd Int. Conf. Int. Conf. Mach. Learn. - Vol. 37 2342\u20132350"},{"key":"1427_CR38","unstructured":"Kaiser \u0141, Sutskever I (2015) Neural GPUs learn algorithms"},{"key":"1427_CR39","unstructured":"Yin W, Kann K, Yu M, Sch\u00fctze H (2017) Comparative study of CNN and RNN for natural language processing"},{"key":"1427_CR40","unstructured":"Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: 32nd international conference on machine learning, ICML 2015. International machine learning society (IMLS), pp 448\u2013456"},{"key":"1427_CR41","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1\u201330","journal-title":"J Mach Learn Res"}],"container-title":["Personal and Ubiquitous Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00779-020-01427-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00779-020-01427-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00779-020-01427-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,30]],"date-time":"2022-10-30T17:04:10Z","timestamp":1667149450000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00779-020-01427-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,27]]},"references-count":41,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,10]]}},"alternative-id":["1427"],"URL":"https:\/\/doi.org\/10.1007\/s00779-020-01427-x","relation":{},"ISSN":["1617-4909","1617-4917"],"issn-type":[{"type":"print","value":"1617-4909"},{"type":"electronic","value":"1617-4917"}],"subject":[],"published":{"date-parts":[[2020,6,27]]},"assertion":[{"value":"1 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 June 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}