{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:45:09Z","timestamp":1760147109871,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,10]],"date-time":"2023-01-10T00:00:00Z","timestamp":1673308800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003052","name":"Korea Institute of Energy Technology Evaluation and Planning (KETEP)","doi-asserted-by":"publisher","award":["20202910100030"],"award-info":[{"award-number":["20202910100030"]}],"id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A haptic sensor coupled to a gamepad or headset is frequently used to enhance the sense of immersion for game players. However, providing haptic feedback for appropriate sound effects involves specialized audio engineering techniques to identify target sounds that vary according to the game. We propose a deep learning-based method for sound event detection (SED) to determine the optimal timing of haptic feedback in extremely noisy environments. To accomplish this, we introduce the BattleSound dataset, which contains a large volume of game sound recordings of game effects and other distracting sounds, including voice chats from a PlayerUnknown\u2019s Battlegrounds (PUBG) game. Given the highly noisy and distracting nature of war-game environments, we set the annotation interval to 0.5 s, which is significantly shorter than the existing benchmarks for SED, to increase the likelihood that the annotated label contains sound from a single source. As a baseline, we adopt mobile-sized deep learning models to perform two tasks: weapon sound event detection (WSED) and voice chat activity detection (VCAD). The accuracy of the models trained on BattleSound was greater than 90% for both tasks; thus, BattleSound enables real-time game sound recognition in noisy environments via deep learning. In addition, we demonstrated that performance degraded significantly when the annotation interval was greater than 0.5 s, indicating that the BattleSound with short annotation intervals is advantageous for SED applications that demand real-time inferences.<\/jats:p>","DOI":"10.3390\/s23020770","type":"journal-article","created":{"date-parts":[[2023,1,10]],"date-time":"2023-01-10T01:57:48Z","timestamp":1673315868000},"page":"770","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["BattleSound: A Game Sound Benchmark for the Sound-Specific Feedback Generation in a Battle Game"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5393-6169","authenticated-orcid":false,"given":"Sungho","family":"Shin","sequence":"first","affiliation":[{"name":"School of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1712-0282","authenticated-orcid":false,"given":"Seongju","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4996-5949","authenticated-orcid":false,"given":"Changhyun","family":"Jun","sequence":"additional","affiliation":[{"name":"School of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4299-4923","authenticated-orcid":false,"given":"Kyoobin","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.ijhcs.2014.09.002","article-title":"Use of auditory event-related potentials to measure immersion during a computer game","volume":"73","author":"Burns","year":"2015","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_2","unstructured":"Stach, T., and Graham, T.C.N. (2011, January 5\u20139). Exploring Haptic Feedback in Exergames. Proceedings of the 13th IFIP TC 13 International Conference on Human-Computer Interaction\u2014Volume Part II, Lisbon, Portugal. INTERACT\u201911."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"102717","DOI":"10.1016\/j.ijhcs.2021.102717","article-title":"A comparison of the effects of haptic and visual feedback on presence in virtual reality","volume":"157","author":"Gibbs","year":"2022","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1007\/BF03392342","article-title":"Game audio\u2014An investigation into the effect of audio on player immersion","volume":"2","author":"Gallacher","year":"2013","journal-title":"Comput. Games J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/97.736233","article-title":"A statistical model-based voice activity detection","volume":"6","author":"Sohn","year":"1999","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"You, Y., Lee, H., Sung, M.Y., Jun, K., and Kang, J.S. (2008, January 18\u201321). Sound-Specific Vibration Interface: Its Performance of Tactile Effects and Applications. Proceedings of the 2008 The 9th International Conference for Young Computer Scientists, Zhangjiajie, China.","DOI":"10.1109\/ICYCS.2008.525"},{"key":"ref_7","unstructured":"Lee, J.U., Lim, J.M., Shin, H., and Kyung, K.U. (2013, January 11\u201314). Haptic Interaction with User Manipulation for Smartphone. Proceedings of the 2013 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"69795","DOI":"10.1109\/ACCESS.2019.2919126","article-title":"Bearing Fault Classification Based on Convolutional Neural Network in Noise Environment","volume":"7","author":"Jiang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"11871","DOI":"10.1109\/ACCESS.2018.2809611","article-title":"An mHealth System for Monitoring Medication Adherence in Obstructive Respiratory Diseases Using Content Based Audio Classification","volume":"6","author":"Nousias","year":"2018","journal-title":"IEEE Access"},{"key":"ref_10","first-page":"535","article-title":"Deep Neural Network for Respiratory Sound Classification in Wearable Devices Enabled by Patient Specific Model Tuning","volume":"14","author":"Acharya","year":"2020","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yu, Y., and Kim, Y.J. (December, January 29). A voice activity detection model composed of bidirectional lstm and attention mechanism. Proceedings of the 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management, HNICEM 2018, Baguio City, Philippines.","DOI":"10.1109\/HNICEM.2018.8666342"},{"key":"ref_12","unstructured":"Cho, Y.D., Al-Naimi, K., and Kondoz, A. (2001, January 7\u201311). Improved voice activity detection based on a smoothed statistical likelihood ratio. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, Salt Lake City, UT, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TASL.2010.2052803","article-title":"Robust voice activity detection using long-term signal variability","volume":"19","author":"Ghosh","year":"2011","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gemmeke, J.F., Ellis, D.P., Freedman, D., Jansen, A., Lawrence, W., Moore, R.C., Plakal, M., and Ritter, M. (2017, January 5\u20139). Audio Set: An ontology and human-labeled dataset for audio events. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952261"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Grollmisch, S., Abeber, J., Liebetrau, J., and Lukashevich, H. (2019, January 2\u20136). Sounding Industry: Challenges and Datasets for Industrial Sound Analysis. Proceedings of the 2019 27th European Signal Processing Conference (EUSIPCO), A Coruna, Spain.","DOI":"10.23919\/EUSIPCO.2019.8902941"},{"key":"ref_16","unstructured":"(2023, January 05). Official homepage of the Playerunknown\u2019s Battleground. Available online: https:\/\/www.pubg.com."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9017","DOI":"10.1109\/ACCESS.2018.2800728","article-title":"A Convolutional Neural Network Smartphone App for Real-Time Voice Activity Detection","volume":"6","author":"Sehgal","year":"2018","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sang, J., Park, S., and Lee, J. (2018, January 3\u20137). Convolutional Recurrent Neural Networks for Urban Sound Classification Using Raw Waveforms. Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy.","DOI":"10.23919\/EUSIPCO.2018.8553247"},{"key":"ref_19","unstructured":"Stowell, D., and Plumbley, M.D. (2014, January 27\u201329). An open dataset for research on audio field recording archives: Freefield1010. Proceedings of the AES International Conference, London, UK."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Koizumi, Y., Saito, S., Uematsu, H., Harada, N., and Imoto, K. (2019, January 20\u201323). ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection. Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2019.8937164"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Foster, P., Sigtia, S., Krstulovic, S., Barker, J., and Plumbley, M.D. (2015, January 18\u201321). Chime-home: A dataset for sound source recognition in a domestic environment. Proceedings of the 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2015.7336899"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Salamon, J., MacConnell, D., Cartwright, M., Li, P., and Bello, J.P. (2017, January 15\u201318). Scaper: A library for soundscape synthesis and augmentation. Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2017.8170052"},{"key":"ref_23","unstructured":"Dekkers, G., Lauwereins, S., Thoen, B., Adhana, M.W., Brouckxon, H., Van den Bergh, B., van Waterschoot, T., Vanrumste, B., Verhelst, M., and Karsmakers, P. (2017, January 16\u201317). The SINS database for detection of daily activities in a home environment using an Acoustic Sensor Network. Proceedings of the DCASE Workshop, Munich, Germany."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Salamon, J., Jacoby, C., and Bello, J.P. (2014, January 3\u20137). A dataset and taxonomy for urban sound research. Proceedings of the MM 2014\u20142014 ACM Conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2655045"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Piczak, K.J. (2015, January 26\u201330). ESC: Dataset for environmental sound classification. Proceedings of the MM 2015\u20142015 ACM Multimedia Conference, Brisbane, Australia.","DOI":"10.1145\/2733373.2806390"},{"key":"ref_26","unstructured":"Mesaros, A., Heittola, T., and Virtanen, T. (2018). A multi-device dataset for urban acoustic scene classification. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19\u201324). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, South Brisbane, QLD, Australia.","DOI":"10.1109\/ICASSP.2015.7178964"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1007\/978-3-319-99579-3_21","article-title":"TED-LIUM 3: Twice as much data and corpus repartition for experiments on speaker adaptation","volume":"Volume 11096","author":"Hernandez","year":"2018","journal-title":"Speech and Computer"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kumar, A., and Raj, B. (2017). Deep CNN framework for audio event recognition using weakly labeled web data. arXiv.","DOI":"10.1145\/2964284.2964310"},{"key":"ref_30","unstructured":"Shah, A., Kumar, A., Hauptmann, A.G., and Raj, B. (2018). A closer look at weak label learning for audio events. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Morehead, A., Ogden, L., Magee, G., Hosler, R., White, B., and Mohler, G. (2019, January 9\u201312). Low Cost Gunshot Detection using Deep Learning on the Raspberry Pi. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.","DOI":"10.1109\/BigData47090.2019.9006456"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1098\/rstb.1992.0070","article-title":"Temporal information in speech: Acoustic, auditory and linguistic aspects","volume":"336","author":"Rosen","year":"1992","journal-title":"Philos. Trans. R. Soc. Lond. Ser. Biol. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/S0167-6393(02)00107-3","article-title":"The analysis of speech in different temporal integration windows: Cerebral lateralization as \u2018asymmetric sampling in time\u2019","volume":"41","author":"Poeppel","year":"2003","journal-title":"Speech Commun."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Renals, S., Hain, T., and Bourlard, H. (2007, January 9\u201313). Recognition and understanding of meetings the ami and amida projects. Proceedings of the 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU), Kyoto, Japan.","DOI":"10.1109\/ASRU.2007.4430116"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F.-F. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1007\/978-3-319-46493-0_32","article-title":"XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks","volume":"Volume 9908","author":"Rastegari","year":"2016","journal-title":"Computer Vision\u2014ECCV 2016"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Arandjelovic, R., and Zisserman, A. (2017, January 22\u201329). Look, Listen and Learn. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.73"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Arandjelovic, R., and Zisserman, A. (2018, January 8\u201314). Objects that Sound. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_27"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1145\/3197517.3201357","article-title":"Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation","volume":"37","author":"Ephrat","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hershey, S., Chaudhuri, S., Ellis, D.P., Gemmeke, J.F., Jansen, A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., and Seybold, B. (2017, January 5\u20139). CNN architectures for large-scale audio classification. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952132"},{"key":"ref_43","unstructured":"Deshmukh, S., Raj, B., and Singh, R. (September, January 30). Improving weakly supervised sound event detection with self-supervised auxiliary tasks. Proceedings of the Interspeech, Brno, Czech Republic."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Park, S., Bellur, A., Han, D.K., and Elhilali, M. (2021, January 6\u201311). Self-Training for Sound Event Detection in Audio Mixtures. Proceedings of the ICASSP 2021\u20142021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9414450"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"104811","DOI":"10.1016\/j.livsci.2021.104811","article-title":"A monophonic cow sound annotation tool using a semi-automatic method on audio\/video data","volume":"256","author":"Pandeya","year":"2022","journal-title":"Livest. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"162625","DOI":"10.1109\/ACCESS.2020.3022058","article-title":"Visual Object Detector for Cow Sound Event Detection","volume":"8","author":"Pandeya","year":"2020","journal-title":"IEEE Access"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Imoto, K., Tonami, N., Koizumi, Y., Yasuda, M., Yamanishi, R., and Yamashita, Y. (2020, January 4\u20138). Sound Event Detection by Multitask Learning of Sound Events and Scenes with Soft Scene Labels. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053912"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Kim, U.H. (September, January 30). Noise-Tolerant Self-Supervised Learning for Audio-Visual Voice Activity Detection. Proceedings of the Interspeech 2021, Brno, Czech Republic.","DOI":"10.21437\/Interspeech.2021-43"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"129329","DOI":"10.1016\/j.foodchem.2021.129329","article-title":"Rapid and Non-Destructive Spectroscopic Method for Classifying Beef Freshness using a Deep Spectral Network Fused with Myoglobin Information","volume":"352","author":"Shin","year":"2021","journal-title":"Food Chem."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Shin, S., Kim, J., Yu, Y., Lee, S., and Lee, K. (2021). Self-Supervised Transfer Learning from Natural Images for Sound Classification. Appl. Sci., 11.","DOI":"10.3390\/app11073043"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_52","unstructured":"Tian, Z., Shen, C., Chen, H., and He, T. (November, January 27). FCOS: Fully Convolutional One-Stage Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Shin, S., Lee, J., Lee, J., Yu, Y., and Lee, K. (2022, January 23\u201327). Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition. Proceedings of the Computer Vision\u2014ECCV 2022: 17th European Conference, Tel Aviv, Israel. Part XII.","DOI":"10.1007\/978-3-031-19775-8_37"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. Proceedings of the International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/770\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:05:04Z","timestamp":1760119504000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/770"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,10]]},"references-count":54,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020770"],"URL":"https:\/\/doi.org\/10.3390\/s23020770","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,1,10]]}}}