{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T06:30:27Z","timestamp":1782282627767,"version":"3.54.5"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2046737"],"award-info":[{"award-number":["2046737"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["Young Investigator Program W911NF2110216"],"award-info":[{"award-number":["Young Investigator Program W911NF2110216"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>With the resurgence of non-contact vital sign sensing due to the COVID-19 pandemic, remote heart-rate monitoring has gained significant prominence. Many existing methods use cameras; however previous work shows a performance loss for darker skin tones. In this paper, we show through light transport analysis that the camera modality is fundamentally biased against darker skin tones. We propose to reduce this bias through multi-modal fusion with a complementary and fairer modality - radar. Through a novel debiasing oriented fusion framework, we achieve performance gains over all tested baselines and achieve skin tone fairness improvements over the RGB modality. That is, the associated Pareto frontier between performance and fairness is improved when compared to the RGB modality. In addition, performance improvements are obtained over the radar-based method, with small trade-offs in fairness. We also open-source the largest multi-modal remote heart-rate estimation dataset of paired camera and radar measurements with a focus on skin tone representation.<\/jats:p>","DOI":"10.1145\/3528223.3530161","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Blending camera and 77 GHz radar sensing for equitable, robust plethysmography"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6197-734X","authenticated-orcid":false,"given":"Alexander","family":"Vilesov","sequence":"first","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9610-0350","authenticated-orcid":false,"given":"Pradyumna","family":"Chari","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4448-3089","authenticated-orcid":false,"given":"Adnan","family":"Armouti","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1079-8656","authenticated-orcid":false,"given":"Anirudh Bindiganavale","family":"Harish","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1830-1402","authenticated-orcid":false,"given":"Kimaya","family":"Kulkarni","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0562-6526","authenticated-orcid":false,"given":"Ananya","family":"Deoghare","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5083-516X","authenticated-orcid":false,"given":"Laleh","family":"Jalilian","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2444-2503","authenticated-orcid":false,"given":"Achuta","family":"Kadambi","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2012.2228263"},{"key":"e_1_2_1_2_1","volume-title":"Contactless Human Activity Analysis","author":"Rahman Ahad Md Atiqur","unstructured":"Md Atiqur Rahman Ahad, Upal Mahbub, and Tauhidur Rahman. 2021. Contactless Human Activity Analysis. Springer."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2912956"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2017.102"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/1523-1747.ep12479191"},{"key":"e_1_2_1_6_1","unstructured":"Consumer Technology Association. 2018. Physical Activity Monitoring for Heart Rate ANSI\/CTA-2065."},{"key":"e_1_2_1_7_1","volume-title":"Oyku Deniz Bozkurt, and Achuta Kadambi.","author":"Ba Yunhao","year":"2021","unstructured":"Yunhao Ba, Zhen Wang, Kerim Doruk Karinca, Oyku Deniz Bozkurt, and Achuta Kadambi. 2021. Overcoming Difficulty in Obtaining Dark-skinned Subjects for Remote-PPG by Synthetic Augmentation. arXiv preprint arXiv:2106.06007 (2021)."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.440"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.3390\/s18051541"},{"key":"e_1_2_1_10_1","volume-title":"Fairness in machine learning. Nips tutorial 1","author":"Barocas Solon","year":"2017","unstructured":"Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2017. Fairness in machine learning. Nips tutorial 1 (2017), 2017."},{"key":"e_1_2_1_11_1","volume-title":"Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29","author":"Bolukbasi Tolga","year":"2016","unstructured":"Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016), 4349--4357."},{"key":"e_1_2_1_12_1","volume-title":"Conference on fairness, accountability and transparency. PMLR, 77--91","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77--91."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2013.114"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01216-8_22"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2013.2266196"},{"key":"e_1_2_1_16_1","volume-title":"Non-contact measurement of heart and respiration rates with a single-chip microwave doppler radar","author":"Droitcour Amy Diane","unstructured":"Amy Diane Droitcour. 2006. Non-contact measurement of heart and respiration rates with a single-chip microwave doppler radar. Stanford University."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMTT.2004.823552"},{"key":"e_1_2_1_18_1","volume-title":"Understanding bland altman analysis. Biochemia medica 25, 2","author":"Giavarina Davide","year":"2015","unstructured":"Davide Giavarina. 2015. Understanding bland altman analysis. Biochemia medica 25, 2 (2015), 141--151."},{"key":"e_1_2_1_19_1","volume-title":"Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis. arXiv preprint arXiv:2109.00412","author":"Han Wei","year":"2021","unstructured":"Wei Han, Hui Chen, and Soujanya Poria. 2021. Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis. arXiv preprint arXiv:2109.00412 (2021)."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5540167"},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Christophe Hurter and Daniel McDuff. 2017. Cardiolens: remote physiological monitoring in a mixed reality environment. In ACM siggraph 2017 emerging technologies. 1--2.","DOI":"10.1145\/3084822.3084834"},{"key":"e_1_2_1_22_1","volume-title":"The appearance of human skin: A survey","author":"Igarashi Takanori","unstructured":"Takanori Igarashi, Ko Nishino, and Shree K Nayar. 2007. The appearance of human skin: A survey. Now Publishers Inc."},{"key":"e_1_2_1_23_1","volume-title":"International conference on machine learning. PMLR, 448--456","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. PMLR, 448--456."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-019-02999-x"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00922"},{"key":"e_1_2_1_26_1","first-page":"493","article-title":"Remote heart rate monitoring method using infrared thermal camera","volume":"11","author":"Kim Yoonkyoung","year":"2018","unstructured":"Yoonkyoung Kim, Yosep Park, Jinman Kim, and Eui Chul Lee. 2018. Remote heart rate monitoring method using infrared thermal camera. International Journal of Engineering Research and Technology 11, 3 (2018), 493--500.","journal-title":"International Journal of Engineering Research and Technology"},{"key":"e_1_2_1_27_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58583-9_24"},{"key":"e_1_2_1_29_1","volume-title":"Measuring pulse rate with a webcam---a non-contact method for evaluating cardiac activity. In 2011 federated conference on computer science and information systems (FedCSIS)","author":"Lewandowska Magdalena","unstructured":"Magdalena Lewandowska, Jacek Rumi\u0144ski, Tomasz Kocejko, and J\u0119drzej Nowak. 2011. Measuring pulse rate with a webcam---a non-contact method for evaluating cardiac activity. In 2011 federated conference on computer science and information systems (FedCSIS). IEEE, 405--410."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1975.9992"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21082732"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2018.00174"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2990438"},{"key":"e_1_2_1_34_1","volume-title":"Heart Rate Meters, and Alarms. ANSI\/AAMI Standard EC13","author":"Monitors Cardiac","year":"2002","unstructured":"Cardiac Monitors. 2002. Heart Rate Meters, and Alarms. ANSI\/AAMI Standard EC13 (2002)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.3390\/s20082171"},{"key":"e_1_2_1_36_1","volume-title":"Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR mHealth and uHealth 7, 3","author":"Nelson Benjamin W","year":"2019","unstructured":"Benjamin W Nelson and Nicholas B Allen. 2019. Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR mHealth and uHealth 7, 3 (2019), e10828."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2947204"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 284--285","author":"Nowara Ewa M","year":"2020","unstructured":"Ewa M Nowara, Daniel McDuff, and Ashok Veeraraghavan. 2020. A meta-analysis of the impact of skin tone and gender on non-contact photoplethysmography measurements. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 284--285."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/BSC.2008.4563262"},{"key":"e_1_2_1_40_1","volume-title":"Advancements in noncontact, multiparameter physiological measurements using a webcam","author":"Poh Ming-Zher","year":"2010","unstructured":"Ming-Zher Poh, Daniel J McDuff, and Rosalind W Picard. 2010. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE transactions on biomedical engineering 58, 1 (2010), 7--11."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMTT.2017.2658567"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMTT.2016.2597824"},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Silonie Sachdeva et al. 2009. Fitzpatrick skin typing: Applications in dermatology. Indian journal of dermatology venereology and leprology 75 1 (2009) 93.","DOI":"10.4103\/0378-6323.45238"},{"key":"e_1_2_1_44_1","volume-title":"Zeeshan Ul Islam, and Yan Cui","author":"Singh Vikas","year":"2019","unstructured":"Vikas Singh, Nishchal K Verma, Zeeshan Ul Islam, and Yan Cui. 2019. Feature learning using stacked autoencoder for shared and multimodal fusion of medical images. In Computational Intelligence: Theories, Applications and Future Directions-Volume I. Springer, 53--66."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2021.3051176"},{"key":"e_1_2_1_46_1","volume-title":"NIPS","volume":"1","author":"Srivastava Nitish","year":"2012","unstructured":"Nitish Srivastava, Ruslan Salakhutdinov, et al. 2012. Multimodal Learning with Deep Boltzmann Machines.. In NIPS, Vol. 1. Citeseer, 2."},{"key":"e_1_2_1_47_1","volume-title":"Remote plethysmographic imaging using ambient light. Optics express 16, 26","author":"Verkruysse Wim","year":"2008","unstructured":"Wim Verkruysse, Lars O Svaasand, and J Stuart Nelson. 2008. Remote plethysmographic imaging using ambient light. Optics express 16, 26 (2008), 21434--21445."},{"key":"e_1_2_1_48_1","volume-title":"Fairness definitions explained. In 2018 ieee\/acm international workshop on software fairness (fairware)","author":"Verma Sahil","unstructured":"Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In 2018 ieee\/acm international workshop on software fairness (fairware). IEEE, 1--7."},{"key":"e_1_2_1_49_1","volume-title":"BIOMEDICAL ENGINEERING 2016 17","author":"Vizbara Vytautas","year":"2013","unstructured":"Vytautas Vizbara. 2013. Comparison of green, blue and infrared light in wrist and forehead photoplethysmography. BIOMEDICAL ENGINEERING 2016 17, 1 (2013)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2016.2609282"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01993"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00894"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-005-5763-2"},{"key":"e_1_2_1_54_1","volume-title":"Eulerian video magnification for revealing subtle changes in the world. ACM transactions on graphics (TOG) 31, 4","author":"Wu Hao-Yu","year":"2012","unstructured":"Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Fr\u00e9do Durand, and William Freeman. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM transactions on graphics (TOG) 31, 4 (2012), 1--8."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2954294"},{"key":"e_1_2_1_56_1","first-page":"17","article-title":"Ppgi: New development in noninvasive and contactless diagnosis of dermal perfusion using near infrared light","volume":"7","author":"Wu T","year":"2003","unstructured":"T Wu. 2003. Ppgi: New development in noninvasive and contactless diagnosis of dermal perfusion using near infrared light. J. GCPD eV 7, 1 (2003), 17--24.","journal-title":"J. GCPD eV"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.407646"},{"key":"e_1_2_1_58_1","volume-title":"Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. arXiv preprint arXiv:1905.02419","author":"Yu Zitong","year":"2019","unstructured":"Zitong Yu, Xiaobai Li, and Guoying Zhao. 2019. Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. arXiv preprint arXiv:1905.02419 (2019)."},{"key":"e_1_2_1_59_1","volume-title":"Manuel Gomez Rogriguez, and Krishna P Gummadi","author":"Zafar Muhammad Bilal","year":"2017","unstructured":"Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. 2017. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics. PMLR, 962--970."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2016.2603342"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3485730.3485932"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530161","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530161","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530161","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:49Z","timestamp":1750186969000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530161"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530161"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530161","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}