{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T09:26:17Z","timestamp":1775899577667,"version":"3.50.1"},"reference-count":46,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,8,25]],"date-time":"2021-08-25T00:00:00Z","timestamp":1629849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The emergence of various types of commercial cameras (compact, high resolution, high angle of view, high speed, and high dynamic range, etc.) has contributed significantly to the understanding of human activities. By taking advantage of the characteristic of a high angle of view, this paper demonstrates a system that recognizes micro-behaviors and a small group discussion with a single 360 degree camera towards quantified meeting analysis. We propose a method that recognizes speaking and nodding, which have often been overlooked in existing research, from a video stream of face images and a random forest classifier. The proposed approach was evaluated on our three datasets. In order to create the first and the second datasets, we asked participants to meet physically: 16 sets of five minutes data from 21 unique participants and seven sets of 10 min meeting data from 12 unique participants. The experimental results showed that our approach could detect speaking and nodding with a macro average f1-score of 67.9% in a 10-fold random split cross-validation and a macro average f1-score of 62.5% in a leave-one-participant-out cross-validation. By considering the increased demand for an online meeting due to the COVID-19 pandemic, we also record faces on a screen that are captured by web cameras as the third dataset and discussed the potential and challenges of applying our ideas to virtual video conferences.<\/jats:p>","DOI":"10.3390\/s21175719","type":"journal-article","created":{"date-parts":[[2021,8,25]],"date-time":"2021-08-25T23:25:50Z","timestamp":1629933950000},"page":"5719","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["DisCaaS: Micro Behavior Analysis on Discussion by Camera as a Sensor"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0252-1785","authenticated-orcid":false,"given":"Ko","family":"Watanabe","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Kaiserslautern & DFKI GmbH, 67663 Kaiserslautern, Germany"}]},{"given":"Yusuke","family":"Soneda","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara 630-0192, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3135-4915","authenticated-orcid":false,"given":"Yuki","family":"Matsuda","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara 630-0192, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8834-5323","authenticated-orcid":false,"given":"Yugo","family":"Nakamura","sequence":"additional","affiliation":[{"name":"Department of Information Science and Technology, Graduate School and Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7156-9160","authenticated-orcid":false,"given":"Yutaka","family":"Arakawa","sequence":"additional","affiliation":[{"name":"Department of Information Science and Technology, Graduate School and Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan"}]},{"given":"Andreas","family":"Dengel","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Kaiserslautern & DFKI GmbH, 67663 Kaiserslautern, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5374-1510","authenticated-orcid":false,"given":"Shoya","family":"Ishimaru","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Kaiserslautern & DFKI GmbH, 67663 Kaiserslautern, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,25]]},"reference":[{"key":"ref_1","first-page":"48","article-title":"Mind Your Meetings: Improve Your Organization\u2019s Effectiveness One Meeting at a Time","volume":"41","author":"Allen","year":"2008","journal-title":"Qual. Prog."},{"key":"ref_2","first-page":"18","article-title":"The Science and Fiction of Meetings","volume":"48","author":"Rogelberg","year":"2007","journal-title":"MIT Sloan Manag. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Romano, N.C., and Nunamaker, J.F. (2001, January 3\u20136). Meeting analysis: Findings from research and practice. Proceedings of the 34th Annual Hawaii International Conference on System Sciences, Maui, HI, USA.","DOI":"10.1109\/HICSS.2001.926253"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Poel, M., Poppe, R., and Nijholt, A. (2008, January 17\u201319). Meeting behavior detection in smart environments: Nonverbal cues that help to obtain natural interaction. Proceedings of the 2008 8th IEEE International Conference on Automatic Face Gesture Recognition, Amsterdam, The Netherlands.","DOI":"10.1109\/AFGR.2008.4813432"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1080\/17447143.2012.685743","article-title":"Meetings: A cultural perspective","volume":"7","author":"Sprain","year":"2012","journal-title":"J. Multicult. Discourses"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1177\/1368430213497066","article-title":"Observing culture: Differences in US-American and German team meeting behaviors","volume":"17","author":"Allen","year":"2014","journal-title":"Group Process. Intergroup Relations"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1177\/0963721418776307","article-title":"Do We Really Need Another Meeting? The Science of Workplace Meetings","volume":"27","author":"Mroz","year":"2018","journal-title":"Curr. Dir. Psychol. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"L\u00fcbstorf, S., and Lehmann-Willenbrock, N. (2020). Are Meetings Really Just Another Stressor? The Relevance of Team Meetings for Individual Well-Being. Research on Managing Groups and Teams, Emerald Publishing Limited.","DOI":"10.1108\/S1534-085620200000020003"},{"key":"ref_9","unstructured":"Schulte, E.M., Lehmann-Willenbrock, N., and Kauffeld, S. (2013). Age, forgiveness, and meeting behavior: A multilevel study. J. Manag. Psychol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1163\/157180805775098595","article-title":"Implementing existing tools: Turning words into actions\u2013Decision-making processes of regional fisheries management organisations (RFMOs)","volume":"20","author":"McDorman","year":"2005","journal-title":"Int. J. Mar. Coast. Law"},{"key":"ref_11","first-page":"73","article-title":"TECHNIQUES TO COMMUNICATE IN VIRTUAL MEETINGS AMIDST THE NEW NORMAL\u2026A CONSIDERATION!!!","volume":"16","author":"Shrivastava","year":"2020","journal-title":"Wutan Huatan Jisuan Jishu"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s10484-006-9014-6","article-title":"The influence of voice volume, pitch, and speech rate on progressive relaxation training: Application of methods from speech pathology and audiology","volume":"31","author":"Knowlton","year":"2006","journal-title":"Appl. Psychophysiol. Biofeedback"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1080\/10810730701508245","article-title":"Public meetings about suspected cancer clusters: The impact of voice, interactional justice, and risk perception on attendees\u2019 attitudes in six communities","volume":"12","author":"McComas","year":"2007","journal-title":"J. Health Commun."},{"key":"ref_14","unstructured":"Williams, J. (2017). Women at Work, Emerald Publishing Limited."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.dcm.2017.09.010","article-title":"Gendering metapragmatics in online discourse: \u201cMansplaining man gonna mansplain\u2026\u201d","volume":"20","author":"Bridges","year":"2017","journal-title":"Discourse Context Media"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1080\/00332747.1964.11023403","article-title":"The significance of posture in communication systems","volume":"27","author":"Scheflen","year":"1964","journal-title":"Psychiatry"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1037\/h0027349","article-title":"Significance of posture and position in the communication of attitude and status relationships","volume":"71","author":"Mehrabian","year":"1969","journal-title":"Psychol. Bull."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.evolhumbehav.2014.08.001","article-title":"Honest signaling in trust interactions: Smiles rated as genuine induce trust and signal higher earning opportunities","volume":"36","author":"Centorrino","year":"2015","journal-title":"Evol. Hum. Behav."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.displa.2012.10.009","article-title":"Eye contact and video-mediated communication: A review","volume":"34","author":"Bohannon","year":"2013","journal-title":"Displays"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1016\/j.pragma.2007.02.009","article-title":"Nodding, aizuchi, and final particles in Japanese conversation: How conversation reflects the ideology of communication and social relationships","volume":"39","author":"Kita","year":"2007","journal-title":"J. Pragmat."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1080\/10463280802402609","article-title":"Forgiveness in personal relationships: Its malleability and powerful consequences","volume":"19","author":"Karremans","year":"2008","journal-title":"Eur. Rev. Soc. Psychol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1177\/1046496411429599","article-title":"Meetings matter: Effects of team meetings on team and organizational success","volume":"43","author":"Kauffeld","year":"2012","journal-title":"Small Group Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2234","DOI":"10.1109\/TPAMI.2007.70733","article-title":"Automatic age estimation based on facial aging patterns","volume":"29","author":"Geng","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","unstructured":"(2021, August 24). The FG-NET Aging Database. Available online: https:\/\/yanweifu.github.io\/FG_NET_data\/."},{"key":"ref_25","unstructured":"Ricanek, K., and Tesafaye, T. (2006, January 10\u201312). Morph: A longitudinal image database of normal adult age-progression. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR06), Southampton, UK."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/S0262-8856(97)00069-3","article-title":"Statistical models of face images\u2014Improving specificity","volume":"16","author":"Edwards","year":"1998","journal-title":"Image Vis. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/j.patrec.2015.06.006","article-title":"A deep analysis on age estimation","volume":"68","author":"Huerta","year":"2015","journal-title":"Pattern Recognit. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hebda, B., and Kryjak, T. (2016, January 11\u201314). A compact deep convolutional neural network architecture for video based age and gender estimation. Proceedings of the 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), Gdansk, Poland.","DOI":"10.15439\/2016F472"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yu, D., and Deng, L. (2016). Automatic Speech Recognition, Springer.","DOI":"10.1007\/978-1-4471-5779-3"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, L., Zhao, Z., Ma, C., Shan, L., Sun, H., Jiang, L., Deng, S., and Gao, C. (2020). End-to-end automatic pronunciation error detection based on improved hybrid ctc\/attention architecture. Sensors, 20.","DOI":"10.3390\/s20071809"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3132027","article-title":"Semi-automated 8 collaborative online training module for improving communication skills","volume":"1","author":"Zhao","year":"2017","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_32","unstructured":"Janin, A., Baron, D., Edwards, J., Ellis, D., Gelbart, D., Morgan, N., Peskin, B., Pfau, T., Shriberg, E., and Stolcke, A. (2003, January 6\u201310). The ICSI Meeting Corpus. Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP \u201903), Hong Kong, China."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kraaij, W., and Kronenthal, M. (2005, January 11\u201313). The AMI Meeting Corpus: A Pre-Announcement. Proceedings of the Second International Conference on Machine Learning for Multimodal Interaction, Edinburgh, UK. MLMI\u201905.","DOI":"10.1007\/11677482_3"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1016\/j.specom.2010.06.002","article-title":"Long story short\u2014Global unsupervised models for keyphrase based meeting summarization","volume":"52","author":"Riedhammer","year":"2010","journal-title":"Speech Commun."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pham, H.H., Salmane, H., Khoudour, L., Crouzil, A., Velastin, S.A., and Zegers, P. (2020). A unified deep framework for joint 3d pose estimation and action recognition from a single rgb camera. Sensors, 20.","DOI":"10.3390\/s20071825"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhang, X., Sugano, Y., and Bulling, A. (2017, January October). Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, Quebec City, QC, Canada. UIST\u201917.","DOI":"10.1145\/3126594.3126614"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1515\/semi.1969.1.1.49","article-title":"The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding","volume":"1","author":"Ekman","year":"1969","journal-title":"Semiotica"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1016\/j.artint.2007.04.003","article-title":"Head gestures for perceptual interfaces: The role of context in improving recognition","volume":"171","author":"Morency","year":"2007","journal-title":"Artif. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yu, Z., Yu, Z., Aoyama, H., Ozeki, M., and Nakamura, Y. (April, January 29). Capture, recognition, and visualization of human semantic interactions in meetings. Proceedings of the 2010 IEEE International Conference on Pervasive Computing and Communications (PerCom), Mannheim, Germany.","DOI":"10.1109\/PERCOM.2010.5466987"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ohnishi, A., Murao, K., Terada, T., and Tsukamoto, M. (2019). A method for structuring meeting logs using wearable sensors. Internet Things, 140\u2013152.","DOI":"10.1016\/j.iot.2019.01.005"},{"key":"ref_41","unstructured":"Ricoh Company, L. (2021, August 24). Product|RICOH THETA V. Available online: https:\/\/theta360.com\/de\/about\/theta\/v.html."},{"key":"ref_42","unstructured":"Google Inc., G. (2021, August 24). Product|GOOGLE MEET. Available online: https:\/\/meet.google.com\/."},{"key":"ref_43","unstructured":"Archive, T.L. (2021, August 24). ELAN. Available online: https:\/\/archive.mpi.nl\/tla\/elan."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tadas Baltru\u0161aitis, P.R., and Morency, L.P. (2016, January 7\u201310). OpenFace: An open source facial behavior analysis toolkit. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Placid, NY, USA.","DOI":"10.1109\/WACV.2016.7477553"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Nakamura, Y., Matsuda, Y., Arakawa, Y., and Yasumoto, K. (2019). WaistonBelt X: A Belt-Type Wearable Device with Sensing and Intervention Toward Health Behavior Change. Sensors, 19.","DOI":"10.3390\/s19204600"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Soneda, Y., Matsuda, Y., Arakawa, Y., and Yasumoto, K. (2019). M3B Corpus: Multi-Modal Meeting Behavior Corpus for Group Meeting Assessment. UbiComp\/ISWC \u201919 Adjunct, Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, Association for Computing Machinery.","DOI":"10.1145\/3341162.3345588"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5719\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:51:30Z","timestamp":1760165490000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5719"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,25]]},"references-count":46,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21175719"],"URL":"https:\/\/doi.org\/10.3390\/s21175719","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,25]]}}}