{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:38:09Z","timestamp":1750307889038,"version":"3.41.0"},"reference-count":24,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2010,7,1]],"date-time":"2010-07-01T00:00:00Z","timestamp":1277942400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["811450"],"award-info":[{"award-number":["811450"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Appl. Percept."],"published-print":{"date-parts":[[2010,7]]},"abstract":"<jats:p>We consume video content in a multitude of ways, including in movie theaters, on television, on DVDs and Blu-rays, online, on smart phones, and on portable media players. For quality control purposes, it is important to have a uniform viewing experience across these various platforms. In this work, we focus on voice synchronization, an aspect of video quality that is strongly affected by current post-production and transmission practices. We examined the synchronization of an actor's voice and lip movements in two distinct scenarios. First, we simulated the temporal mismatch between the audio and video tracks that can occur during dubbing or during broadcast. Next, we recreated the pitch changes that result from conversions between formats with different frame rates. We show, for the first time, that these audio visual mismatches affect viewer enjoyment. When temporal synchronization is noticeably absent, there is a decrease in the perceived performance quality and the perceived emotional intensity of a performance. For pitch changes, we find that higher pitch voices are not preferred, especially for male actors. Based on our findings, we advise that mismatched audio and video signals negatively affect viewer experience.<\/jats:p>","DOI":"10.1145\/1823738.1823741","type":"journal-article","created":{"date-parts":[[2010,7,22]],"date-time":"2010-07-22T18:52:11Z","timestamp":1279824731000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Perceptually motivated guidelines for voice synchronization in film"],"prefix":"10.1145","volume":"7","author":[{"given":"Elizabeth J.","family":"Carter","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lavanya","family":"Sharan","sequence":"additional","affiliation":[{"name":"Disney Research Pittsburgh, Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Laura","family":"Trutoiu","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Iain","family":"Matthews","sequence":"additional","affiliation":[{"name":"Disney Research Pittsburgh, Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jessica K.","family":"Hodgins","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2010,7,26]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 7th International Conference on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems.","author":"Abelin A.","year":"2007","unstructured":"Abelin , A. 2007 . Emotional McGurk effect\u2014An experiment . In Proceedings of the 7th International Conference on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems. Abelin, A. 2007. Emotional McGurk effect\u2014An experiment. In Proceedings of the 7th International Conference on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1163\/156856897X00357"},{"volume-title":"Audio-Vision: Sound on Screen","author":"Chion M.","key":"e_1_2_1_3_1","unstructured":"Chion , M. 1994. Audio-Vision: Sound on Screen . Columbia University Press . Chion, M. 1994. Audio-Vision: Sound on Screen. Columbia University Press."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1068\/p090719"},{"key":"e_1_2_1_5_1","volume-title":"Human Face: Guidelines for Research and an Integration of Findings","author":"Ekman P.","year":"1972","unstructured":"Ekman , P. , Friesen , W. V. , and Ellsworth , P . 1972 . Emotion in the Human Face: Guidelines for Research and an Integration of Findings . Pergamon Press . Ekman, P., Friesen, W. V., and Ellsworth, P. 1972. Emotion in the Human Face: Guidelines for Research and an Integration of Findings. Pergamon Press."},{"key":"e_1_2_1_6_1","first-page":"1","article-title":"Darpa timit acoustic-phonetic continuous speech corpus","author":"Garofolo J.","year":"1993","unstructured":"Garofolo , J. , Lamel , L. , Fisher , W. , Fiscus , F. , Pallett , D. , and Dahlgren , N. 1993 . Darpa timit acoustic-phonetic continuous speech corpus . Published on CD-ROM: NIST Speech Disc 1 - 1 .1, Oct. 1990 NISTIR 4930, National Institute of Standards and Technology, Gaithersburg, MD. Garofolo, J., Lamel, L., Fisher, W., Fiscus, F., Pallett, D., and Dahlgren, N. 1993. Darpa timit acoustic-phonetic continuous speech corpus. Published on CD-ROM: NIST Speech Disc 1-1.1, Oct. 1990 NISTIR 4930, National Institute of Standards and Technology, Gaithersburg, MD.","journal-title":"Published on CD-ROM: NIST Speech Disc"},{"volume-title":"Proceedings of the Auditory-Visual Speech Processing Conference.","author":"Grant K. W.","key":"e_1_2_1_7_1","unstructured":"Grant , K. W. and Greenberg , S . 2001. Speech intelligibility derived from asynchronous processing of auditory-visual information . In Proceedings of the Auditory-Visual Speech Processing Conference. Grant, K. W. and Greenberg, S. 2001. Speech intelligibility derived from asynchronous processing of auditory-visual information. In Proceedings of the Auditory-Visual Speech Processing Conference."},{"volume-title":"Proceedings of the Auditory-Visual Speech Processing Conference.","author":"Grant K. W.","key":"e_1_2_1_8_1","unstructured":"Grant , K. W. , van Wassenhove , V. , and Poeppel , D . 2003. Discriminatory of auditory-visual synchrony . In Proceedings of the Auditory-Visual Speech Processing Conference. Grant, K. W., van Wassenhove, V., and Poeppel, D. 2003. Discriminatory of auditory-visual synchrony. In Proceedings of the Auditory-Visual Speech Processing Conference."},{"key":"e_1_2_1_9_1","unstructured":"Huang E. Sisk J. Kirk T. Coryell G. and Stewart J. 2007. Searching for an ideal live video streaming technology. http:\/\/www.iupui.edu\/~nmstream\/live\/introduction.php.  Huang E. Sisk J. Kirk T. Coryell G. and Stewart J. 2007. Searching for an ideal live video streaming technology. http:\/\/www.iupui.edu\/~nmstream\/live\/introduction.php."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.380958"},{"volume-title":"Proceedings of the Audio Engineering Society Convention. Audio Engineering Society.","author":"Mason A.","key":"e_1_2_1_11_1","unstructured":"Mason , A. and Salmon , R . 2008. Factors affecting perception of audio-video synchronization in television . In Proceedings of the Audio Engineering Society Convention. Audio Engineering Society. Mason, A. and Salmon, R. 2008. Factors affecting perception of audio-video synchronization in television. In Proceedings of the Audio Engineering Society Convention. Audio Engineering Society."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1038\/264746a0"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03206811"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1163\/156856897X00366"},{"key":"e_1_2_1_15_1","unstructured":"Phillips G. 2010. Personal communication.  Phillips G. 2010. Personal communication."},{"volume-title":"Proceedings of the Auditory-Visual Speech Processing Conference.","author":"Sakamoto S.","key":"e_1_2_1_16_1","unstructured":"Sakamoto , S. , Tanaka , A. , Tsumura , K. , and Suzuki , Y . 2007. Effect of speed difference between time-expanded speech and talker's moving image on word or sentence intelligibility . In Proceedings of the Auditory-Visual Speech Processing Conference. Sakamoto, S., Tanaka, A., Tsumura, K., and Suzuki, Y. 2007. Effect of speed difference between time-expanded speech and talker's moving image on word or sentence intelligibility. In Proceedings of the Auditory-Visual Speech Processing Conference."},{"key":"e_1_2_1_17_1","unstructured":"Spill H. 2010. Personal communication.  Spill H. 2010. Personal communication."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.1907309"},{"volume-title":"Some Preliminaries to a Comprehensive Account of Audio-Visual Speech Perception","author":"Summerfield Q.","key":"e_1_2_1_19_1","unstructured":"Summerfield , Q. 1987. Some Preliminaries to a Comprehensive Account of Audio-Visual Speech Perception . Lawrence Erlbaum Associates , London . Summerfield, Q. 1987. Some Preliminaries to a Comprehensive Account of Audio-Visual Speech Perception. Lawrence Erlbaum Associates, London."},{"key":"e_1_2_1_20_1","article-title":"Lip reading and audio-visual speech perception","volume":"335","author":"Summerfield Q.","year":"1992","unstructured":"Summerfield , Q. 1992 . Lip reading and audio-visual speech perception . Philosoph. Trans. Biol. Sci. 335 , 1273, 71--78. Summerfield, Q. 1992. Lip reading and audio-visual speech perception. Philosoph. Trans. Biol. Sci. 335, 1273, 71--78.","journal-title":"Philosoph. Trans. Biol. Sci."},{"volume-title":"Proceedings of the 7th International Conference on Audio Visual Speech Processing.","author":"Theobald B.-J.","key":"e_1_2_1_21_1","unstructured":"Theobald , B.-J. , Bangham , A. , Matthews , I. , and Cawley , G . 2003. Evaluation of a talking head based on appearance models . In Proceedings of the 7th International Conference on Audio Visual Speech Processing. Theobald, B.-J., Bangham, A., Matthews, I., and Cawley, G. 2003. Evaluation of a talking head based on appearance models. In Proceedings of the 7th International Conference on Audio Visual Speech Processing."},{"key":"e_1_2_1_22_1","unstructured":"Tinwell A. and Grimshaw M. 2009. Survival horror games: An uncanny modality. In Thinking After Dark.  Tinwell A. and Grimshaw M. 2009. Survival horror games: An uncanny modality. In Thinking After Dark."},{"key":"e_1_2_1_23_1","unstructured":"Tuckman J. Chrisafis A. Hooper J. Watts J. Smee J. and Ramesh R. 2006. To dub you have to be as good an actor. Or better. http:\/\/www.guardian.co.uk\/film\/2006\/nov\/03\/3.  Tuckman J. Chrisafis A. Hooper J. Watts J. Smee J. and Ramesh R. 2006. To dub you have to be as good an actor. Or better. http:\/\/www.guardian.co.uk\/film\/2006\/nov\/03\/3."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuropsychologia.2006.01.001"}],"container-title":["ACM Transactions on Applied Perception"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1823738.1823741","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1823738.1823741","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T14:47:16Z","timestamp":1750258036000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1823738.1823741"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,7]]},"references-count":24,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2010,7]]}},"alternative-id":["10.1145\/1823738.1823741"],"URL":"https:\/\/doi.org\/10.1145\/1823738.1823741","relation":{},"ISSN":["1544-3558","1544-3965"],"issn-type":[{"type":"print","value":"1544-3558"},{"type":"electronic","value":"1544-3965"}],"subject":[],"published":{"date-parts":[[2010,7]]},"assertion":[{"value":"2010-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2010-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2010-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}