{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,11]],"date-time":"2026-01-11T12:04:02Z","timestamp":1768133042676,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":44,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,10,10]],"date-time":"2022-10-10T00:00:00Z","timestamp":1665360000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,10,10]]},"DOI":"10.1145\/3503161.3547850","type":"proceedings-article","created":{"date-parts":[[2022,10,10]],"date-time":"2022-10-10T15:42:35Z","timestamp":1665416555000},"page":"2042-2050","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Speech Fusion to Face: Bridging the Gap Between Human's Vocal Characteristics and Facial Imaging"],"prefix":"10.1145","author":[{"given":"Yeqi","family":"BAI","sequence":"first","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Ma","sequence":"additional","affiliation":[{"name":"Shanghai AI Laboratory, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lipo","family":"Wang","sequence":"additional","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenjie","family":"Zhang","sequence":"additional","affiliation":[{"name":"Neuron Mobility Pte. Ltd., Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,10,10]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240578"},{"key":"e_1_3_2_2_2_1","volume-title":"International conference on machine learning. 1247--1255","author":"Andrew Galen","year":"2013","unstructured":"Galen Andrew , Raman Arora , Jeff Bilmes , and Karen Livescu . 2013 . Deep canonical correlation analysis . In International conference on machine learning. 1247--1255 . Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. 2013. Deep canonical correlation analysis. In International conference on machine learning. 1247--1255."},{"key":"e_1_3_2_2_3_1","volume-title":"StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018","author":"Choi Yunjey","year":"2018","unstructured":"Yunjey Choi , Min-Je Choi , Munyoung Kim , Jung-Woo Ha , Sunghun Kim , and Jaegul Choo . 2018 . StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018 , Salt Lake City, UT, USA, June 18--22 , 2018. 8789--8797. Yunjey Choi, Min-Je Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18--22, 2018. 8789--8797."},{"key":"e_1_3_2_2_4_1","volume-title":"StarGAN v2: Diverse Image Synthesis for Multiple Domains. CoRR","author":"Choi Yunjey","year":"1865","unstructured":"Yunjey Choi , Youngjung Uh , Jaejun Yoo , and Jung-Woo Ha. 2019. StarGAN v2: Diverse Image Synthesis for Multiple Domains. CoRR , Vol. abs\/ 1912 .0 1865 (2019). Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. 2019. StarGAN v2: Diverse Image Synthesis for Multiple Domains. CoRR, Vol. abs\/1912.01865 (2019)."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"crossref","unstructured":"Joon Son Chung Arsha Nagrani and Andrew Zisserman. 2018. VoxCeleb2: Deep Speaker Recognition. In Interspeech. 1086--1090.  Joon Son Chung Arsha Nagrani and Andrew Zisserman. 2018. VoxCeleb2: Deep Speaker Recognition. In Interspeech. 1086--1090.","DOI":"10.21437\/Interspeech.2018-1929"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.367"},{"key":"e_1_3_2_2_7_1","volume-title":"Synthesizing Normalized Faces From Facial Identity Features. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Cole Forrester","unstructured":"Forrester Cole , David Belanger , Dilip Krishnan , Aaron Sarna , Inbar Mosseri , and William T. Freeman . 2017 . Synthesizing Normalized Faces From Facial Identity Features. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Forrester Cole, David Belanger, Dilip Krishnan, Aaron Sarna, Inbar Mosseri, and William T. Freeman. 2017. Synthesizing Normalized Faces From Facial Identity Features. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.96"},{"key":"e_1_3_2_2_9_1","unstructured":"Xavier Giro Amanda Duarte Francisco Roldan Miquel Tubau Janna Escur Santiago Pascual Amaia Salvador Eva Mohedano Kevin McGuinness Jordi Torres etal 2019. Wav2Pix: Speech-conditioned Face Generation using Generative Adversarial Networks. (2019).  Xavier Giro Amanda Duarte Francisco Roldan Miquel Tubau Janna Escur Santiago Pascual Amaia Salvador Eva Mohedano Kevin McGuinness Jordi Torres et al. 2019. Wav2Pix: Speech-conditioned Face Generation using Generative Adversarial Networks. (2019)."},{"key":"e_1_3_2_2_10_1","volume-title":"Advances in Neural Information Processing Systems 27","author":"Goodfellow Ian","unstructured":"Ian Goodfellow , Jean Pouget-Abadie , Mehdi Mirza , Bing Xu , David Warde-Farley , Sherjil Ozair , Aaron Courville , and Yoshua Bengio . 2014. Generative Adversarial Nets . In Advances in Neural Information Processing Systems 27 , Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc. , 2672--2680. http:\/\/papers.nips.cc\/paper\/5423-generative-adversarial-nets.pdf Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2672--2680. http:\/\/papers.nips.cc\/paper\/5423-generative-adversarial-nets.pdf"},{"key":"e_1_3_2_2_11_1","volume-title":"Putting a face to the voice: Fusing audio and visual signals across a video to determine speakers. arXiv preprint arXiv:1706.00079","author":"Hoover Ken","year":"2017","unstructured":"Ken Hoover , Sourish Chaudhuri , Caroline Pantofaru , Malcolm Slaney , and Ian Sturdy . 2017. Putting a face to the voice: Fusing audio and visual signals across a video to determine speakers. arXiv preprint arXiv:1706.00079 ( 2017 ). Ken Hoover, Sourish Chaudhuri, Caroline Pantofaru, Malcolm Slaney, and Ian Sturdy. 2017. Putting a face to the voice: Fusing audio and visual signals across a video to determine speakers. arXiv preprint arXiv:1706.00079 (2017)."},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240601"},{"key":"e_1_3_2_2_13_1","volume-title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 ( 2015 ). Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00133"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073658"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_2_2_18_1","volume-title":"Asian Conference on Computer Vision. Springer, 276--292","author":"Kim Changil","year":"2018","unstructured":"Changil Kim , Hijung Valentina Shin , Tae-Hyun Oh , Alexandre Kaspar , Mohamed Elgharib , and Wojciech Matusik . 2018 . On learning associations of faces and voices . In Asian Conference on Computer Vision. Springer, 276--292 . Changil Kim, Hijung Valentina Shin, Tae-Hyun Oh, Alexandre Kaspar, Mohamed Elgharib, and Wojciech Matusik. 2018. On learning associations of faces and voices. In Asian Conference on Computer Vision. Springer, 276--292."},{"key":"e_1_3_2_2_19_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba . 2015 . Adam : A Method for Stochastic Optimization. In ICLR. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR."},{"key":"e_1_3_2_2_20_1","volume-title":"Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114","author":"Kingma Diederik P","year":"2013","unstructured":"Diederik P Kingma and Max Welling . 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 ( 2013 ). Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)."},{"key":"e_1_3_2_2_21_1","volume-title":"Fundamentals of music processing: Audio, analysis, algorithms, applications","author":"M\u00fcller Meinard","unstructured":"Meinard M\u00fcller . 2015. Fundamentals of music processing: Audio, analysis, algorithms, applications . Springer . Meinard M\u00fcller. 2015. Fundamentals of music processing: Audio, analysis, algorithms, applications .Springer."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_5"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00879"},{"key":"e_1_3_2_2_24_1","volume-title":"Joon Son Chung, and Andrew Zisserman","author":"Nagrani Arsha","year":"2017","unstructured":"Arsha Nagrani , Joon Son Chung, and Andrew Zisserman . 2017 . VoxCeleb: A Large-Scale Speaker Identification Dataset. In Interspeech . 2616--2620. Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. VoxCeleb: A Large-Scale Speaker Identification Dataset. In Interspeech. 2616--2620."},{"key":"e_1_3_2_2_25_1","unstructured":"Jiquan Ngiam Aditya Khosla Mingyu Kim Juhan Nam Honglak Lee and Andrew Y Ng. 2011. Multimodal deep learning. (2011).  Jiquan Ngiam Aditya Khosla Mingyu Kim Juhan Nam Honglak Lee and Andrew Y Ng. 2011. Multimodal deep learning. (2011)."},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00772"},{"key":"e_1_3_2_2_27_1","volume-title":"Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759","author":"van den Oord Aaron","year":"2016","unstructured":"Aaron van den Oord , Nal Kalchbrenner , and Koray Kavukcuoglu . 2016. Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759 ( 2016 ). Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. 2016. Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759 (2016)."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"crossref","unstructured":"Omkar M Parkhi Andrea Vedaldi and Andrew Zisserman. 2015. Deep face recognition. (2015).  Omkar M Parkhi Andrea Vedaldi and Andrew Zisserman. 2015. Deep face recognition. (2015).","DOI":"10.5244\/C.29.41"},{"key":"e_1_3_2_2_29_1","volume-title":"Advances in Neural Information Processing Systems 29","author":"Pu Yunchen","unstructured":"Yunchen Pu , Zhe Gan , Ricardo Henao , Xin Yuan , Chunyuan Li , Andrew Stevens , and Lawrence Carin . 2016. Variational Autoencoder for Deep Learning of Images, Labels and Captions . In Advances in Neural Information Processing Systems 29 , D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (Eds.). Curran Associates, Inc. , 2352--2360. http:\/\/papers.nips.cc\/paper\/6528-variational-autoencoder-for-deep-learning-of-images-labels-and-captions.pdf Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. 2016. Variational Autoencoder for Deep Learning of Images, Labels and Captions. In Advances in Neural Information Processing Systems 29, D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (Eds.). Curran Associates, Inc., 2352--2360. http:\/\/papers.nips.cc\/paper\/6528-variational-autoencoder-for-deep-learning-of-images-labels-and-captions.pdf"},{"key":"e_1_3_2_2_30_1","volume-title":"Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434","author":"Radford Alec","year":"2015","unstructured":"Alec Radford , Luke Metz , and Soumith Chintala . 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 ( 2015 ). Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)."},{"key":"e_1_3_2_2_31_1","volume-title":"Speech-driven expressive talking lips with conditional sequential generative adversarial networks","author":"Sadoughi Najmeh","year":"2019","unstructured":"Najmeh Sadoughi and Carlos Busso . 2019. Speech-driven expressive talking lips with conditional sequential generative adversarial networks . IEEE Transactions on Affective Computing ( 2019 ). Najmeh Sadoughi and Carlos Busso. 2019. Speech-driven expressive talking lips with conditional sequential generative adversarial networks. IEEE Transactions on Affective Computing (2019)."},{"key":"e_1_3_2_2_32_1","unstructured":"Tim Salimans Ian Goodfellow Wojciech Zaremba Vicki Cheung Alec Radford and Xi Chen. 2016. Improved techniques for training gans. In Advances in neural information processing systems. 2234--2242.  Tim Salimans Ian Goodfellow Wojciech Zaremba Vicki Cheung Alec Radford and Xi Chen. 2016. Improved techniques for training gans. In Advances in neural information processing systems. 2234--2242."},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2019.8683477"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073640"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073699","article-title":"A deep learning approach for generalized speech animation","volume":"36","author":"Taylor Sarah","year":"2017","unstructured":"Sarah Taylor , Taehwan Kim , Yisong Yue , Moshe Mahler , James Krahe , Anastasio Garcia Rodriguez , Jessica Hodgins , and Iain Matthews . 2017 . A deep learning approach for generalized speech animation . ACM Transactions on Graphics (TOG) , Vol. 36 , 4 (2017), 1 -- 11 . Sarah Taylor, Taehwan Kim, Yisong Yue, Moshe Mahler, James Krahe, Anastasio Garcia Rodriguez, Jessica Hodgins, and Iain Matthews. 2017. A deep learning approach for generalized speech animation. ACM Transactions on Graphics (TOG), Vol. 36, 4 (2017), 1--11.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Li Wan Quan Wang Alan Papir and Ignacio Lopez-Moreno. 2018. Generalized End-to-End Loss for Speaker Verification. In ICASSP. 4879--4883.  Li Wan Quan Wang Alan Papir and Ignacio Lopez-Moreno. 2018. Generalized End-to-End Loss for Speaker Verification. In ICASSP. 4879--4883.","DOI":"10.1109\/ICASSP.2018.8462665"},{"key":"e_1_3_2_2_39_1","volume-title":"Weiyang Liu, Bhiksha Raj, and Rita Singh.","author":"Wen Yandong","year":"2018","unstructured":"Yandong Wen , Mahmoud Al Ismail , Weiyang Liu, Bhiksha Raj, and Rita Singh. 2018 . Disjoint mapping network for cross-modal matching of voices and faces. arXiv preprint arXiv:1807.04836 (2018). Yandong Wen, Mahmoud Al Ismail, Weiyang Liu, Bhiksha Raj, and Rita Singh. 2018. Disjoint mapping network for cross-modal matching of voices and faces. arXiv preprint arXiv:1807.04836 (2018)."},{"key":"e_1_3_2_2_40_1","unstructured":"Yandong Wen Bhiksha Raj and Rita Singh. 2019. Face Reconstruction from Voice using Generative Adversarial Networks. In Advances in Neural Information Processing Systems. 5266--5275.  Yandong Wen Bhiksha Raj and Rita Singh. 2019. Face Reconstruction from Voice using Generative Adversarial Networks. In Advances in Neural Information Processing Systems. 5266--5275."},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_41"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00143"},{"key":"e_1_3_2_2_43_1","volume-title":"Pastegan: A semi-parametric method to generate image from scene graph. In Advances in Neural Information Processing Systems. 3950--3960.","author":"Yikang LI","year":"2019","unstructured":"LI Yikang , Tao Ma , Yeqi Bai , Nan Duan , Sining Wei , and Xiaogang Wang . 2019 . Pastegan: A semi-parametric method to generate image from scene graph. In Advances in Neural Information Processing Systems. 3950--3960. LI Yikang, Tao Ma, Yeqi Bai, Nan Duan, Sining Wei, and Xiaogang Wang. 2019. Pastegan: A semi-parametric method to generate image from scene graph. In Advances in Neural Information Processing Systems. 3950--3960."},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.629"}],"event":{"name":"MM '22: The 30th ACM International Conference on Multimedia","location":"Lisboa Portugal","acronym":"MM '22","sponsor":["SIGMM ACM Special Interest Group on Multimedia"]},"container-title":["Proceedings of the 30th ACM International Conference on Multimedia"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503161.3547850","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3503161.3547850","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:35Z","timestamp":1750186955000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503161.3547850"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,10]]},"references-count":44,"alternative-id":["10.1145\/3503161.3547850","10.1145\/3503161"],"URL":"https:\/\/doi.org\/10.1145\/3503161.3547850","relation":{},"subject":[],"published":{"date-parts":[[2022,10,10]]},"assertion":[{"value":"2022-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}