{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T16:23:30Z","timestamp":1782577410683,"version":"3.54.5"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,3,29]]},"abstract":"<jats:p>Camera-based contactless photoplethysmography refers to a set of popular techniques for contactless physiological measurement. The current state-of-the-art neural models are typically trained in a supervised manner using videos accompanied by gold standard physiological measurements. However, they often generalize poorly out-of-domain examples (i.e., videos that are unlike those in the training set). Personalizing models can help improve model generalizability, but many personalization techniques still require some gold standard data. To help alleviate this dependency, in this paper, we present a novel mobile sensing system called MobilePhys, the first mobile personalized remote physiological sensing system, that leverages both front and rear cameras on a smartphone to generate high-quality self-supervised labels for training personalized contactless camera-based PPG models. To evaluate the robustness of MobilePhys, we conducted a user study with 39 participants who completed a set of tasks under different mobile devices, lighting conditions\/intensities, motion tasks, and skin types. Our results show that MobilePhys significantly outperforms the state-of-the-art on-device supervised training and few-shot adaptation methods. Through extensive user studies, we further examine how does MobilePhys perform in complex real-world settings. We envision that calibrated or personalized camera-based contactless PPG models generated from our proposed dual-camera mobile sensing system will open the door for numerous future applications such as smart mirrors, fitness and mobile health applications.<\/jats:p>","DOI":"10.1145\/3517225","type":"journal-article","created":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T13:42:46Z","timestamp":1648561366000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["MobilePhys"],"prefix":"10.1145","volume":"6","author":[{"given":"Xin","family":"Liu","sequence":"first","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuntao","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sinan","family":"Xie","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zixian","family":"Ma","sequence":"additional","affiliation":[{"name":"Zhejiang University, HangZhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"McDuff","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shwetak","family":"Patel","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,3,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.440"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.2291073"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2017.10.017"},{"key":"e_1_2_1_4_1","volume-title":"Diverse R-PPG: Camera-Based Heart Rate Estimation for Diverse Subject Skin-Tones and Scenes. arXiv preprint arXiv:2010.12769","author":"Chari Pradyumna","year":"2020","unstructured":"Pradyumna Chari, Krish Kabra, Doruk Karinca, Soumyarup Lahiri, Diplav Srivastava, Kimaya Kulkarni, Tianyuan Chen, Maxime Cannesson, Laleh Jalilian, and Achuta Kadambi. 2020. Diverse R-PPG: Camera-Based Heart Rate Estimation for Diverse Subject Skin-Tones and Scenes. arXiv preprint arXiv:2010.12769 (2020)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01216-8_22"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00100"},{"key":"e_1_2_1_7_1","volume-title":"Accuracy of smartphone apps for heart rate measurement. European journal of preventive cardiology 24, 12","author":"Coppetti Thomas","year":"2017","unstructured":"Thomas Coppetti, Andreas Brauchlin, Simon M\u00fcggler, Adrian Attinger-Toller, Christian Templin, Felix Sch\u00f6nrath, Jens Hellermann, Thomas F L\u00fcscher, Patric Biaggi, and Christophe A Wyss. 2017. Accuracy of smartphone apps for heart rate measurement. European journal of preventive cardiology 24, 12 (2017), 1287--1293."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2013.2266196"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2013.2266196"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1088\/0967-3334\/35\/9\/1913"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2014.6974121"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305498"},{"key":"e_1_2_1_13_1","volume-title":"The validity and practicality of sun-reactive skin types I through VI. Archives of dermatology 124, 6","author":"Fitzpatrick Thomas B","year":"1988","unstructured":"Thomas B Fitzpatrick. 1988. The validity and practicality of sun-reactive skin types I through VI. Archives of dermatology 124, 6 (1988), 869--871."},{"key":"e_1_2_1_14_1","volume-title":"Artem Molchanov, Franziska Meier, Douwe Kiela, Kyunghyun Cho, and Soumith Chintala.","author":"Grefenstette Edward","year":"2019","unstructured":"Edward Grefenstette, Brandon Amos, Denis Yarats, Phu Mon Htut, Artem Molchanov, Franziska Meier, Douwe Kiela, Kyunghyun Cho, and Soumith Chintala. 2019. Generalized inner loop meta-learning. arXiv preprint arXiv:1910.01727 (2019)."},{"key":"e_1_2_1_15_1","volume-title":"Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439","author":"Hospedales Timothy","year":"2020","unstructured":"Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2020. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439 (2020)."},{"key":"e_1_2_1_16_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58583-9_24"},{"key":"e_1_2_1_18_1","volume-title":"Timothy Tipoe, Tong Liu, Martin CS Wong, Aaron Jesuthasan, Adrian Baranchuk, Gary Tse, and Bryan P Yan.","author":"Christien Li Ka Hou","year":"2019","unstructured":"Ka Hou Christien Li, Francesca Anne White, Timothy Tipoe, Tong Liu, Martin CS Wong, Aaron Jesuthasan, Adrian Baranchuk, Gary Tse, and Bryan P Yan. 2019. The current state of mobile phone apps for monitoring heart rate, heart rate variability, and atrial fibrillation: narrative review. JMIR mHealth and uHealth 7, 2 (2019), e11606."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.543"},{"key":"e_1_2_1_20_1","volume-title":"Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835","author":"Li Zhenguo","year":"2017","unstructured":"Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. 2017. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835 (2017)."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00718"},{"key":"e_1_2_1_22_1","volume-title":"Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement. arXiv preprint arXiv:2006.03790","author":"Liu Xin","year":"2020","unstructured":"Xin Liu, Josh Fromm, Shwetak Patel, and Daniel McDuff. 2020. Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement. arXiv preprint arXiv:2006.03790 (2020)."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450439.3451870"},{"key":"e_1_2_1_24_1","volume-title":"Estepp","author":"McDuff Daniel","year":"2017","unstructured":"Daniel McDuff, Ethan B. Blackford, and Justin R. Estepp. 2017. The Impact of Video Compression on Remote Cardiac Pulse Measurement Using Imaging Photoplethysmography. In IEEE FG. https:\/\/www.microsoft.com\/en-us\/research\/publication\/the-impact-of-video-compression-on-remote-cardiac-pulse-measurement-using-imaging-photoplethysmography\/"},{"key":"e_1_2_1_25_1","volume-title":"Advancing Non-Contact Vital Sign Measurement using Synthetic Avatars. arXiv preprint arXiv:2010.12949","author":"McDuff Daniel","year":"2020","unstructured":"Daniel McDuff, Javier Hernandez, Erroll Wood, Xin Liu, and Tadas Baltrusaitis. 2020. Advancing Non-Contact Vital Sign Measurement using Synthetic Avatars. arXiv preprint arXiv:2010.12949 (2020)."},{"key":"e_1_2_1_26_1","unstructured":"Daniel Mcduff Student Member Sarah Gontarek and Rosalind W. Picard. [n.d.]. 1Improvements in Remote Cardio-Pulmonary Measurement Using a Five Band Digital Camera."},{"key":"e_1_2_1_27_1","volume-title":"A survey of remote optical photoplethysmographic imaging methods. In 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC)","author":"McDuff Daniel J","unstructured":"Daniel J McDuff, Justin R Estepp, Alyssa M Piasecki, and Ethan B Blackford. 2015. A survey of remote optical photoplethysmographic imaging methods. In 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, 6398--6404."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2947204"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 284--285","author":"Nowara Ewa M","year":"2020","unstructured":"Ewa M Nowara, Daniel McDuff, and Ashok Veeraraghavan. 2020. A Meta-Analysis of the Impact of Skin Tone and Gender on Non-Contact Photoplethysmography Measurements. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 284--285."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_35"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2048259.2048261"},{"key":"e_1_2_1_32_1","volume-title":"Advancements in noncontact, multiparameter physiological measurements using a webcam","author":"Poh Ming-Zher","year":"2010","unstructured":"Ming-Zher Poh, Daniel J McDuff, and Rosalind W Picard. 2010. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE transactions on biomedical engineering 58, 1 (2010), 7--11."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1364\/OE.18.010762"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-74955-4"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2011.2163157"},{"key":"e_1_2_1_36_1","volume-title":"Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). Journal of telemedicine and telecare","author":"Smith Anthony C","year":"2020","unstructured":"Anthony C Smith, Emma Thomas, Centaine L Snoswell, Helen Haydon, Ateev Mehrotra, Jane Clemensen, and Liam J Caffery. 2020. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). Journal of telemedicine and telecare (2020), 1357633X20916567."},{"key":"e_1_2_1_37_1","unstructured":"Jake Snell Kevin Swersky and Richard Zemel. 2017. Prototypical networks for few-shot learning. In Advances in neural information processing systems. 4077--4087."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2020.2984168"},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Xuan Song Xinyan Liu and Chunting Wang. 2020. The role of telemedicine during the COVID-19 epidemic in China---experience from Shandong province.","DOI":"10.1186\/s13054-020-02884-9"},{"key":"e_1_2_1_40_1","unstructured":"Radim Spetlik Vojtech Franc Jan Cech and Jiri Matas. 2018. Visual Heart Rate Estimation with Convolutional Neural Network. In BMVC."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1088\/0967-3334\/35\/5\/807"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1364\/OE.16.021434"},{"key":"e_1_2_1_43_1","volume-title":"Non-contact physiological monitoring of preterm infants in the Neonatal Intensive Care Unit. npj Digital Medicine 2, 1","author":"Villarroel Mauricio","year":"2019","unstructured":"Mauricio Villarroel, Sitthichok Chaichulee, Jo\u00e3o Jorge, Sara Davis, Gabrielle Green, Carlos Arteta, Andrew Zisserman, Kenny McCormick, Peter Watkinson, and Lionel Tarassenko. 2019. Non-contact physiological monitoring of preterm infants in the Neonatal Intensive Care Unit. npj Digital Medicine 2, 1 (2019), 1--18."},{"key":"e_1_2_1_44_1","unstructured":"Oriol Vinyals Charles Blundell Timothy Lillicrap Daan Wierstra et al. 2016. Matching networks for one shot learning. In Advances in neural information processing systems. 3630--3638."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2016.2609282"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2016.2609282"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2014.2356291"},{"key":"e_1_2_1_48_1","unstructured":"Zitong Yu Xiaobai Li and Guoying Zhao. 2019. Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks. arXiv:1905.02419 [cs.CV]"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3517225","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3517225","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:24:45Z","timestamp":1752467085000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3517225"}},"subtitle":["Personalized Mobile Camera-Based Contactless Physiological Sensing"],"short-title":[],"issued":{"date-parts":[[2022,3,29]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3,29]]}},"alternative-id":["10.1145\/3517225"],"URL":"https:\/\/doi.org\/10.1145\/3517225","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,29]]},"assertion":[{"value":"2022-03-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}