{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:12:20Z","timestamp":1760058740201,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,4,24]],"date-time":"2025-04-24T00:00:00Z","timestamp":1745452800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Psychological safety is pivotal for co-creation to build an open environment where innovative ideas can flourish. Traditionally, psychological safety has been evaluated from a stable and long-term perspective by implementing psychological scales. Consequently, existing interventions often focus on steadily enhancing psychological safety, which is less suitable for dynamic short-term co-creation settings. The purpose of this study is to introduce the use of emojis as a novel and intuitive interaction during co-creations and assess their effectiveness in evaluating and influencing psychological safety. We performed two experiments with 140 participants in total to test emojis as evaluations and interventions, respectively. The participants watched videos and annotated them with emojis based on their perceptions of emotions. This process allowed us to explore the relationship between perceived emotions and psychological safety. In the next phase, we embedded emojis directly into the videos to observe whether the participants\u2019 emotional perceptions\u2014and, consequently, their psychological safety\u2014could be influenced by visual cues. Our findings demonstrate that positive emojis are positively correlated with psychological safety and negative emojis are negatively correlated with psychological safety. We also revealed that negative emojis significantly decreased psychological safety scores, whereas positive emojis did not lead to a corresponding increase.<\/jats:p>","DOI":"10.3390\/mti9050040","type":"journal-article","created":{"date-parts":[[2025,4,24]],"date-time":"2025-04-24T06:18:02Z","timestamp":1745475482000},"page":"40","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring an Emoji-Based Evaluation and Intervention Method for Psychological Safety in Ongoing Co-Creations"],"prefix":"10.3390","volume":"9","author":[{"given":"Qiner","family":"Lyu","sequence":"first","affiliation":[{"name":"Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa 277-0882, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6572-2755","authenticated-orcid":false,"given":"Gaku","family":"Kutsuzawa","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Kashiwa 277-0882, Japan"}]},{"given":"Hiroyuki","family":"Umemura","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Kashiwa 277-0882, Japan"}]},{"given":"Kenta","family":"Kimura","sequence":"additional","affiliation":[{"name":"Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa 277-0882, Japan"},{"name":"National Institute of Advanced Industrial Science and Technology, Kashiwa 277-0882, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4147-853X","authenticated-orcid":false,"given":"Masaaki","family":"Mochimaru","sequence":"additional","affiliation":[{"name":"Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa 277-0882, Japan"},{"name":"National Institute of Advanced Industrial Science and Technology, Kashiwa 277-0882, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2035-4346","authenticated-orcid":false,"given":"Akihiko","family":"Murai","sequence":"additional","affiliation":[{"name":"Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa 277-0882, Japan"},{"name":"National Institute of Advanced Industrial Science and Technology, Kashiwa 277-0882, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"350","DOI":"10.2307\/2666999","article-title":"Psychological safety and learning behavior in work teams","volume":"44","author":"Edmondson","year":"1999","journal-title":"Adm. Sci. Q."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1146\/annurev-orgpsych-120920-055217","article-title":"Psychological safety comes of age: Observed themes in an established literature","volume":"10","author":"Edmondson","year":"2023","journal-title":"Annu. Rev. Organ. Psychol. Organ. Behav."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"O\u2019Donovan, R., and McAuliffe, E. (2020). A systematic review exploring the content and outcomes of interventions to improve psychological safety, speaking up and voice behaviour. BMC Health Serv. Res., 20.","DOI":"10.1186\/s12913-020-4931-2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"O\u2019Donovan, R., De Br\u00fan, A., and McAuliffe, E. (2021). Healthcare professionals experience of psychological safety, voice, and silence. Front. Psychol., 12.","DOI":"10.3389\/fpsyg.2021.626689"},{"key":"ref_5","first-page":"425","article-title":"Exploring the role of psychological safety in promoting the intention to continue sharing knowledge in virtual communities","volume":"30","author":"Zhang","year":"2010","journal-title":"Int. J. Inf. Manag."},{"key":"ref_6","first-page":"521","article-title":"Psychological safety: A systematic review of the literature","volume":"27","author":"Newman","year":"2017","journal-title":"Hum. Resour. Manag. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1146\/annurev-orgpsych-031413-091305","article-title":"Psychological safety: The history, renaissance, and future of an interpersonal construct","volume":"1","author":"Edmondson","year":"2014","journal-title":"Annu. Rev. Organ. Psychol. Organ. Behav."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1111\/peps.12183","article-title":"Psychological safety: A meta-analytic review and extension","volume":"70","author":"Frazier","year":"2017","journal-title":"Pers. Psychol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1023\/A:1019637632584","article-title":"Understanding self-report bias in organizational behavior research","volume":"17","author":"Donaldson","year":"2002","journal-title":"J. Bus. Psychol."},{"key":"ref_10","unstructured":"Hoenderdos, J.W. (2013). Towards an Observational Measure for Team Psychological Safety, University of Twente."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1097\/SIH.0b013e31818cffd3","article-title":"Teaching residents the two-challenge rule: A simulation-based approach to improve education and patient safety","volume":"4","author":"Simon","year":"2009","journal-title":"Simul. Healthc."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1097\/ACM.0000000000001033","article-title":"Improving anesthesiologists\u2019 ability to speak up in the operating room: A randomized controlled experiment of a simulation-based intervention and a qualitative analysis of hurdles and enablers","volume":"91","author":"Raemer","year":"2016","journal-title":"Acad. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1038\/sj.jp.7211771","article-title":"Teaching teamwork during the Neonatal Resuscitation Program: A randomized trial","volume":"27","author":"Thomas","year":"2007","journal-title":"J. Perinatol."},{"key":"ref_14","first-page":"426","article-title":"\u2018Excuse me:\u2019 teaching interns to speak up","volume":"39","author":"Byrne","year":"2013","journal-title":"Jt. Comm. J. Qual. Patient Saf."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1097\/NNA.0b013e31826a1e8a","article-title":"A strategy to improve nurses speaking up and collaborating for patient safety","volume":"42","author":"Sayre","year":"2012","journal-title":"J. Nurs. Adm."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liebeskind, C., and Liebeskind, S. (2019). Emoji Prediction for Hebrew Political Domain. Companion Proceedings of the 2019 World Wide Web Conference, Association for Computing Machinery.","DOI":"10.1145\/3308560.3316548"},{"key":"ref_17","first-page":"265","article-title":"Multilingual emoji prediction using BERT for sentiment analysis","volume":"16","author":"Tomihira","year":"2020","journal-title":"Int. J. Web Inf. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/j.foodqual.2017.04.009","article-title":"Dominant meanings of facial emoji: Insights from Chinese consumers and comparison with meanings from internet resources","volume":"62","author":"Jaeger","year":"2017","journal-title":"Food Qual. Prefer."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kralj Novak, P., Smailovi\u0107, J., Sluban, B., and Mozeti\u010d, I. (2015). Sentiment of emojis. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0144296"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kutsuzawa, G., Umemura, H., Eto, K., and Kobayashi, Y. (2022). Classification of 74 facial emoji\u2019s emotional states on the valence-arousal axes. Sci. Rep., 12.","DOI":"10.1038\/s41598-021-04357-7"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.foodqual.2015.04.008","article-title":"Can consumer segmentation in projective mapping contribute to a better understanding of consumer perception?","volume":"47","author":"Vidal","year":"2016","journal-title":"Food Qual. Prefer."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1016\/j.foodqual.2017.12.011","article-title":"Emoji questionnaires can be used with a range of population segments: Findings relating to age, gender and frequency of emoji\/emoticon use","volume":"68","author":"Jaeger","year":"2018","journal-title":"Food Qual. Prefer."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1016\/j.asoc.2017.07.056","article-title":"Consumer insight mining: Aspect based Twitter opinion mining of mobile phone reviews","volume":"68","author":"Rathan","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1177\/1069072717748647","article-title":"Contextualizing interest scales with emojis: Implications for measurement and validity","volume":"27","author":"Phan","year":"2019","journal-title":"J. Career Assess."},{"key":"ref_25","first-page":"182","article-title":"An alternative to Likert scale: Emoji","volume":"12","author":"Uysal","year":"2021","journal-title":"J. Meas. Eval. Educ. Psychol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1177\/1046496408328703","article-title":"Group size, group development, and group productivity","volume":"40","author":"Wheelan","year":"2009","journal-title":"Small Group Res."},{"key":"ref_27","first-page":"97","article-title":"Changes in workplace practices during the COVID-19 pandemic: The roles of emotion, psychological safety and organisation support","volume":"8","author":"Lee","year":"2021","journal-title":"J. Organ. Eff. People Perform."},{"key":"ref_28","first-page":"924","article-title":"Big Five Content Representation of the Japanese Version of the Ten-Item Personality Inventory","volume":"4","author":"Oshio","year":"2013","journal-title":"Sci. Res."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1080\/15298868.2016.1269668","article-title":"Counteractive control over temptations: Promoting resistance through enhanced perception of conflict and goal value","volume":"16","author":"Ozaki","year":"2017","journal-title":"Self Identity"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1111\/j.1467-6486.2009.00825.x","article-title":"Do East Asians respond more\/less strongly to organizational justice than North Americans? A meta-analysis","volume":"46","author":"Li","year":"2009","journal-title":"J. Manag. Stud."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"692","DOI":"10.2307\/256287","article-title":"Psychological conditions of personal engagement and disengagement at work","volume":"33","author":"Kahn","year":"1990","journal-title":"Acad. Manag. J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"183","DOI":"10.3991\/ijet.v18i08.37825","article-title":"The effect of virtual reality game teaching technology on students\u2019 immersion","volume":"18","author":"Jing","year":"2023","journal-title":"Int. J. Emerg. Technol. Learn. (Online)"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1207\/S15327957PSPR0504_2","article-title":"Negativity bias, negativity dominance, and contagion","volume":"5","author":"Rozin","year":"2001","journal-title":"Pers. Soc. Psychol. Rev."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1037\/1089-2680.5.4.323","article-title":"Bad is stronger than good","volume":"5","author":"Baumeister","year":"2001","journal-title":"Rev. Gen. Psychol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1037\/0022-3514.75.4.887","article-title":"Negative information weighs more heavily on the brain: The negativity bias in evaluative categorizations","volume":"75","author":"Ito","year":"1998","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1037\/0022-3514.90.2.210","article-title":"Being bad isn\u2019t always good: Affective context moderates the attention bias toward negative information","volume":"90","author":"Smith","year":"2006","journal-title":"J. Pers. Soc. Psychol."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/5\/40\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:20:39Z","timestamp":1760030439000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/5\/40"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,24]]},"references-count":37,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["mti9050040"],"URL":"https:\/\/doi.org\/10.3390\/mti9050040","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2025,4,24]]}}}