{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:41:26Z","timestamp":1760366486919,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,1,9]],"date-time":"2020-01-09T00:00:00Z","timestamp":1578528000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and ICT","award":["20190000500012004"],"award-info":[{"award-number":["20190000500012004"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Inspired by spiders that can generate and sense vibrations to obtain information regarding a substrate, we propose an intelligent system that can recognize the type of surface being touched by knocking the surface and listening to the vibrations. Hence, we developed a system that is equipped with an electromagnetic hammer for hitting the ground and an accelerometer for measuring the mechanical responses induced by the impact. We investigate the feasibility of sensing 10 different daily surfaces through various machine-learning techniques including recent deep-learning approaches. Although some test surfaces are similar, experimental results show that our system can recognize 10 different surfaces remarkably well (test accuracy of 98.66%). In addition, our results without directly hitting the surface (internal impact) exhibited considerably high test accuracy (97.51%). Finally, we conclude this paper with the limitations and future directions of the study.<\/jats:p>","DOI":"10.3390\/s20020369","type":"journal-article","created":{"date-parts":[[2020,1,9]],"date-time":"2020-01-09T03:07:11Z","timestamp":1578539231000},"page":"369","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Knocking and Listening: Learning Mechanical Impulse Response for Understanding Surface Characteristics"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4183-0014","authenticated-orcid":false,"given":"Semin","family":"Ryu","sequence":"first","affiliation":[{"name":"Intelligent Robotics Laboratory, Hallym University, Chuncheon 24252, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7292-5166","authenticated-orcid":false,"given":"Seung-Chan","family":"Kim","sequence":"additional","affiliation":[{"name":"Intelligent Robotics Laboratory, Hallym University, Chuncheon 24252, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1590\/S1519-566X2004000200001","article-title":"Vibrational communication in insects","volume":"33","author":"Cokl","year":"2004","journal-title":"Neotrop. Entomol."},{"key":"ref_2","first-page":"1215","article-title":"Vibrational communication and the ecology of group-living, herbivorous insects","volume":"41","author":"Cocroft","year":"2001","journal-title":"Am. Zool."},{"key":"ref_3","first-page":"827","article-title":"Signal transmission in natural environments","volume":"8","author":"Wiley","year":"2009","journal-title":"New Encycl. Neurosci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Barth, F.G. (1985). Neuroethology of the spider vibration sense. Neurobiology of Arachnids, Springer.","DOI":"10.1007\/978-3-642-70348-5"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Harrison, C., Schwarz, J., and Hudson, S.E. (2011, January 16\u201319). TapSense: Enhancing Finger Interaction on Touch Surfaces. Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA.","DOI":"10.1145\/2047196.2047279"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Harrison, C., and Hudson, S.E. (2008, January 19\u201322). Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces. Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA.","DOI":"10.1145\/1449715.1449747"},{"key":"ref_7","unstructured":"Heo, S., and Lee, G. (September, January 30). Forcetap: Extending the Input Vocabulary of Mobile Touch Screens by Adding Tap Gestures. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, New York, NY, USA."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ono, M., Shizuki, B., and Tanaka, J. (2013, January 8\u201311). Touch & Activate: Adding Interactivity to Existing Objects Using Active Acoustic Sensing. Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA.","DOI":"10.1145\/2501988.2501989"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s001380050130","article-title":"An automatic assessment scheme for steel quality inspection","volume":"12","author":"Wiltschi","year":"2000","journal-title":"Mach. Vis. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1016\/S0031-3203(03)00005-0","article-title":"Neural network based detection of local textile defects","volume":"36","author":"Kumar","year":"2003","journal-title":"Pattern Recogn."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1007\/s00138-002-0084-z","article-title":"Wood inspection with non-supervised clustering","volume":"13","author":"Niskanen","year":"2003","journal-title":"Mach. Vis. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/s10044-004-0232-3","article-title":"Detection of surface defects on raw steel blocks using Bayesian network classifiers","volume":"7","author":"Pernkopf","year":"2004","journal-title":"Pattern Anal. Appl."},{"key":"ref_13","first-page":"13","article-title":"Restructured eigenfilter matching for novelty detection in random textures","volume":"5","author":"Monadjemi","year":"2004","journal-title":"Learning"},{"key":"ref_14","first-page":"9","article-title":"Non-contact measurement system for the type of floor surface recognition using discrete optical methods","volume":"56","author":"Tarapata","year":"2015","journal-title":"Elektron. Konstr. Technol. Zastos."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"17501","DOI":"10.3390\/s131217501","article-title":"Floor Covering and Surface Identification for Assistive Mobile Robotic Real-Time Room Localization Application","volume":"13","author":"Gillham","year":"2013","journal-title":"Sensors"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chandra, G.R., Sathya, G., Rajan, E.G., and Coyle, M.P. (2014, January 5\u20138). Surface Detection in 3D images using Cellular Logic Array Processing. Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), San Diego, CA, USA.","DOI":"10.1109\/SMC.2014.6974017"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Aujeszky, T., Korres, G., and Eid, M. (2017, January 22\u201323). Thermography-based material classification using machine learning. Proceedings of the 2017 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), Abu Dhabi, UAE.","DOI":"10.1109\/HAVE.2017.8240344"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1080\/17686733.2018.1539895","article-title":"Material classification with laser thermography and machine learning","volume":"16","author":"Aujeszky","year":"2019","journal-title":"Quant. Infrared Thermogr. J."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kleyko, D., Hostettler, R., Birk, W., and Osipov, E. (2015, January 15\u201318). Comparison of machine learning techniques for vehicle classification using road side sensors. Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Las Palmas, Spain.","DOI":"10.1109\/ITSC.2015.100"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tarapata, G., Paczesny, D., and Tarasiuk, \u0141. (2016, January 19\u201322). Electronic system for floor surface type detection in robotics applications. Proceedings of the 14th International Conference on Optical and Electronic Sensors, International Society for Optics and Photonics, Gdansk, Poland.","DOI":"10.1117\/12.2249331"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e1700578","DOI":"10.1126\/sciadv.1700578","article-title":"Convolutional neural network for earthquake detection and location","volume":"4","author":"Perol","year":"2018","journal-title":"Sci. Adv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sato, M., Poupyrev, I., and Harrison, C. (2012, January 5\u201310). Touch\u00e9: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA.","DOI":"10.1145\/2207676.2207743"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Han, B.K., Ryu, J.K., and Kim, S.C. (2019). Context-Aware Winter Sports Based on Multivariate Sequence Learning. Sensors, 19.","DOI":"10.3390\/s19153296"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kunze, K., and Lukowicz, P. (2007, January 16\u201319). Symbolic object localization through active sampling of acceleration and sound signatures. Proceedings of the International Conference on Ubiquitous Computing, Innsbruck, Austria.","DOI":"10.1007\/978-3-540-74853-3_10"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cho, J., Hwang, I., and Oh, S. (2012, January 19\u201322). Vibration-based surface recognition for smartphones. Proceedings of the 2012 IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, Seoul, Korea.","DOI":"10.1109\/RTCSA.2012.42"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Laput, G., Xiao, R., and Harrison, C. (2016, January 16\u201319). Viband: High-fidelity bio-acoustic sensing using commodity smartwatch accelerometers. Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan.","DOI":"10.1145\/2984511.2984582"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shi, L., Ashoori, M., Zhang, Y., and Azenkot, S. (2018, January 3\u20136). Knock knock, what\u2019s there: converting passive objects into customizable smart controllers. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, Barcelona, Spain.","DOI":"10.1145\/3229434.3229453"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3351240","article-title":"Knocker: Vibroacoustic-based Object Recognition with Smartphones","volume":"3","author":"Gong","year":"2019","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.patrec.2005.08.011","article-title":"Random Forests for land cover classification","volume":"27","author":"Gislason","year":"2006","journal-title":"Pattern Recogn. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Levantesi, S., and Pizzorusso, V. (2019). Application of Machine Learning to Mortality Modeling and Forecasting. Risks, 7.","DOI":"10.3390\/risks7010026"},{"key":"ref_31","first-page":"1995","article-title":"Convolutional networks for images, speech, and time series","volume":"Volume 3361","author":"LeCun","year":"1995","journal-title":"The Handbook of Brain Theory and Neural Networks"},{"key":"ref_32","unstructured":"Lee, S., Yoon, S.M., and Cho, H. (2017, January 13\u201316). Human activity recognition from accelerometer data using Convolutional Neural Network. Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, Korea."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhu, P., Zhou, H., Cao, S., Yang, P., and Xue, S. (2018, January 7\u20139). Control with Gestures: A Hand Gesture Recognition System Using Off-the-Shelf Smartwatch. Proceedings of the 2018 4th International Conference on Big Data Computing and Communications (BIGCOM), Chicago, IL, USA.","DOI":"10.1109\/BIGCOM.2018.00018"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hwang, W., and Lim, S.C. (2017). Inferring interaction force from visual information without using physical force sensors. Sensors, 17.","DOI":"10.3390\/s17112455"},{"key":"ref_39","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/2\/369\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:04:18Z","timestamp":1760364258000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/2\/369"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,9]]},"references-count":39,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20020369"],"URL":"https:\/\/doi.org\/10.3390\/s20020369","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,1,9]]}}}