{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:20:24Z","timestamp":1760239224585,"version":"build-2065373602"},"reference-count":26,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T00:00:00Z","timestamp":1603065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>In this study, we designed a four-dimensional (4D) audiovisual entertainment system called Sense. This system comprises a scene recognition system and hardware modules that provide haptic sensations for users when they watch movies and animations at home. In the scene recognition system, we used Google Cloud Vision to detect common scene elements in a video, such as fire, explosions, wind, and rain, and further determine whether the scene depicts hot weather, rain, or snow. Additionally, for animated videos, we applied deep learning with a single shot multibox detector to detect whether the animated video contained scenes of fire-related objects. The hardware module was designed to provide six types of haptic sensations set as line-symmetry to provide a better user experience. After the system considers the results of object detection via the scene recognition system, the system generates corresponding haptic sensations. The system integrates deep learning, auditory signals, and haptic sensations to provide an enhanced viewing experience.<\/jats:p>","DOI":"10.3390\/sym12101718","type":"journal-article","created":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T20:44:41Z","timestamp":1603140281000},"page":"1718","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Design of Desktop Audiovisual Entertainment System with Deep Learning and Haptic Sensations"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3673-4324","authenticated-orcid":false,"given":"Chien-Hsing","family":"Chou","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Tamkang University, New Taipei City 25137, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu-Sheng","family":"Su","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, National Taiwan Ocean University, Keelung 20224, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Che-Ju","family":"Hsu","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Tamkang University, New Taipei City 25137, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kong-Chang","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Tamkang University, New Taipei City 25137, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ping-Hsuan","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Interaction Design, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,19]]},"reference":[{"key":"ref_1","first-page":"3231","article-title":"Impact of different sensory stimuli on presence in credible virtual environments","volume":"26","author":"Melo","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"unstructured":"Chen, Y.-S., Han, P.-H., Hsiao, J.-C., Lee, K.-C., Hsieh, C.-E., Lu, K.-Y., Chou, C.-H., and Hung, Y.-P. (2016, January 16\u201319). SoEs: Attachable Augmented Haptic on Gaming Controller for Immersive Interaction. Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan.","key":"ref_2"},{"doi-asserted-by":"crossref","unstructured":"Han, P.H., Chen, Y.S., Yang, K.T., Chuan, W.S., Chang, Y.T., Yang, T.M., Lin, J.Y., Lee, K.C., Hsieh, C.E., and Lee, L.C. (2017, January 27\u201330). BoEs: Attachable Haptics Bits On Gaming Controller For Designing Interactive Gameplay. Proceedings of the ACM SIGGRAPH Asia 2017 VR Showcase, Bangkok, Thailand. Article No. 3.","key":"ref_3","DOI":"10.1145\/3139468.3139474"},{"unstructured":"Han, P.-H., Hsieh, C.-E., Chen, Y.-S., Hsiao, J.-C., Lee, K.-C., Ko, S.F., Chen, K.W., Chou, C.-H., and Hung, Y.-P. (August, January 30). AoEs: Enhancing teleportation experience in immersive environment with mid-air haptics. Proceedings of the ACM SIGGRAPH 2017 Emerging Technologies, Los Angeles, CA, USA. Article No. 3.","key":"ref_4"},{"doi-asserted-by":"crossref","unstructured":"G\u00fcnther, S., M\u00fcller, F., Sch\u00f6n, D., Elmoghazy, O., M\u00fchlh\u00e4user, M., and Schmitz, M. (2020, January 25\u201330). Therminator: Understanding the Interdependency of Visual and On-Body Thermal Feedback in Virtual Reality. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","key":"ref_5","DOI":"10.1145\/3313831.3376195"},{"doi-asserted-by":"crossref","unstructured":"Sasaki, T., Liu, K.-H., Hasegawa, T., Hiyama, A., and Inami, M. (2019, January 11\u201312). Virtual Super-Leaping: Immersive Extreme Jumping in VR. Proceedings of the 10th Augmented Human International Conference 2019 (AH2019), Reims, France. Article No. 18.","key":"ref_6","DOI":"10.1145\/3311823.3311861"},{"unstructured":"(2019, November 24). TensorFlow Object Detection API. Available online: https:\/\/github.com\/TensorFlow\/models\/tree\/master\/research\/object_detection.","key":"ref_7"},{"unstructured":"(2018, September 08). Google Cloud Vision. Available online: https:\/\/cloud.google.com\/vision\/docs\/2018.","key":"ref_8"},{"unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. arXiv.","key":"ref_9"},{"unstructured":"Bergman, D.L. (2019, January 7\u201312). Symmetry Constrained Machine Learning. Proceedings of the 2019 Intelligent Systems and Applications Conference, Las Palmas, Spain.","key":"ref_10"},{"doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","key":"ref_11","DOI":"10.1109\/CVPR.2016.91"},{"doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","key":"ref_12","DOI":"10.1109\/CVPR.2017.690"},{"unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.","key":"ref_13"},{"doi-asserted-by":"crossref","unstructured":"Xu, K., Wang, X., Liu, X., Cao, C., Li, H., Peng, H., and Wang, D. (2020). A dedicated hardware accelerator for real-time acceleration of YOLOv2. J. Real Time Image Process.","key":"ref_14","DOI":"10.1007\/s11554-020-00977-w"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1007\/s11554-018-0780-1","article-title":"Realtime multi-aircraft tracking in aerial scene with deep orientation network","volume":"15","author":"Maher","year":"2018","journal-title":"J. Real Time Image Process."},{"unstructured":"Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., and Zhang, Z. (2016, January 5\u201310). MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. Proceedings of the Advances in Neural Information Processing System, Workshop on Machine Learning System, Barcelona, Spain.","key":"ref_16"},{"doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., and Reed, S. (2016, January 11\u201314). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","key":"ref_17","DOI":"10.1007\/978-3-319-46448-0_2"},{"doi-asserted-by":"crossref","unstructured":"Ko, S.F., Lin, Y.H., Han, P.H., Chang, C.C., and Chou, C.H. (2018, January 4\u20137). Combining Deep Learning Algorithm with Scene Recognition and Haptic Feedback for 4D\u2013VR Cinema. Proceedings of the ACM SIGGRAPH Asia 2018, Tokyo, Japan. Article No. 18.","key":"ref_18","DOI":"10.1145\/3283289.3283321"},{"unstructured":"(2017, October 19). Fusion 360. Available online: https:\/\/www.autodesk.com\/products\/fusion-360\/overview.","key":"ref_19"},{"unstructured":"(2020, August 25). Operating Video of Our System Based on Google Could Vision. Available online: https:\/\/youtu.be\/r-iIezN6quI.","key":"ref_20"},{"unstructured":"(2020, August 25). Operating Video of Our System Based on SSD Deep Learning Algorithm. Available online: https:\/\/youtu.be\/2-XtgCypyYA.","key":"ref_21"},{"unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.","key":"ref_22"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"116529","DOI":"10.1109\/ACCESS.2019.2936143","article-title":"A finger-worn device for exploring Chinese printed text with using CNN algorithm on a micro IoT processor","volume":"7","author":"Su","year":"2019","journal-title":"IEEE ACCESS"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1108\/LHT-01-2019-0028","article-title":"Bibliometric study of social network analysis literature","volume":"38","author":"Su","year":"2019","journal-title":"Libr. Hi Tech."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.3389\/fpsyg.2020.01166","article-title":"Social Facebook with Big Six approaches for improved students\u2019 learning performance and behavior: A case study of a project innovation and implementation course","volume":"11","author":"Su","year":"2020","journal-title":"Front. Psychol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"106298","DOI":"10.1016\/j.asoc.2020.106298","article-title":"Applying deep learning algorithms to enhance simulations of large-scale groundwater flow in IoTs","volume":"92","author":"Su","year":"2020","journal-title":"Appl. Soft Comput."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/10\/1718\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:23:38Z","timestamp":1760178218000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/10\/1718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,19]]},"references-count":26,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["sym12101718"],"URL":"https:\/\/doi.org\/10.3390\/sym12101718","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2020,10,19]]}}}