{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T16:24:35Z","timestamp":1781713475532,"version":"3.54.5"},"reference-count":40,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T00:00:00Z","timestamp":1718236800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"PROTECT COVID-19 National Core Study on transmission and environment, managed by the Health and Safety Executive on behalf of HM Government"},{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","award":["N510129"],"award-info":[{"award-number":["N510129"]}],"id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/S026347\/1]"],"award-info":[{"award-number":["EP\/S026347\/1]"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Through our respiratory system, many viruses and diseases frequently spread and pass from one person to another. Covid-19 served as an example of how crucial it is to track down and cut back on contacts to stop its spread. There is a clear gap in finding automatic methods that can detect hand-to-face contact in complex urban scenes or indoors. In this paper, we introduce a computer vision framework, called FaceTouch, based on deep learning. It comprises deep sub-models to detect humans and analyse their actions. FaceTouch seeks to detect hand-to-face touches in the wild, such as through video chats, bus footage, or CCTV feeds. Despite partial occlusion of faces, the introduced system learns to detect face touches from the RGB representation of a given scene by utilising the representation of the body gestures such as arm movement. This has been demonstrated to be useful in complex urban scenarios beyond simply identifying hand movement and its closeness to faces. Relying on Supervised Contrastive Learning, the introduced model is trained on our collected dataset, given the absence of other benchmark datasets. The framework shows a strong validation in unseen datasets which opens the door for potential deployment.<\/jats:p>","DOI":"10.1371\/journal.pone.0288670","type":"journal-article","created":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T17:44:29Z","timestamp":1718300669000},"page":"e0288670","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious diseases"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7733-7777","authenticated-orcid":true,"given":"Mohamed R.","family":"Ibrahim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Terry","family":"Lyons","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,6,13]]},"reference":[{"key":"pone.0288670.ref001","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.2466\/pms.1980.50.3c.1083","article-title":"Self-Touching, Performance, and Attentional Processes","volume":"50","author":"F Barroso","year":"1980","journal-title":"Percept Mot Skills"},{"key":"pone.0288670.ref002","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3478129","article-title":"FaceSense: Sensing Face Touch with an Ear-worn System","volume":"5","author":"V Kakaraparthi","year":"2021","journal-title":"Proc ACM Interact Mob Wearable Ubiquitous Technol"},{"key":"pone.0288670.ref003","unstructured":"Chen X \u201cAnthony.\u201d FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer. arXiv; 2020. Available: http:\/\/arxiv.org\/abs\/2008.01769"},{"key":"pone.0288670.ref004","doi-asserted-by":"crossref","first-page":"612392","DOI":"10.3389\/frobt.2021.612392","article-title":"FaceGuard: A Wearable System To Avoid Face Touching","volume":"8","author":"AM Michelin","year":"2021","journal-title":"Front Robot AI"},{"key":"pone.0288670.ref005","doi-asserted-by":"crossref","first-page":"102481","DOI":"10.1016\/j.cities.2019.102481","article-title":"Understanding cities with machine eyes: A review of deep computer vision in urban analytics","volume":"96","author":"MR Ibrahim","year":"2020","journal-title":"Cities"},{"key":"pone.0288670.ref006","first-page":"1","article-title":"Cycling near misses: a review of the current methods, challenges and the potential of an AI-embedded system","author":"MR Ibrahim","year":"2020","journal-title":"Transp Rev"},{"key":"pone.0288670.ref007","article-title":"CyclingNet: Detecting cycling near misses from video streams in complex urban scenes with deep learning","author":"MR Ibrahim","year":"2021","journal-title":"IET Intell Transp Syst"},{"key":"pone.0288670.ref008","doi-asserted-by":"crossref","first-page":"549","DOI":"10.3390\/ijgi8120549","article-title":"WeatherNet: Recognising Weather and Visual Conditions from Street-Level Images Using Deep Residual Learning","volume":"8","author":"MR Ibrahim","year":"2019","journal-title":"ISPRS Int J Geo-Inf"},{"key":"pone.0288670.ref009","article-title":"URBAN-i: From urban scenes to mapping slums, transport modes, and pedestrians in cities using deep learning and computer vision","author":"MR Ibrahim","year":"2019","journal-title":"Environ Plan B Urban Anal City Sci"},{"key":"pone.0288670.ref010","doi-asserted-by":"crossref","unstructured":"Nojavanasghari B, Hughes CE, Baltrusaitis T, Morency L. Hand2Face: Automatic Synthesis and Recognition of Hand Over Face Occlusions. arXiv; 2017. Available: http:\/\/arxiv.org\/abs\/1708.00370","DOI":"10.1109\/ACII.2017.8273602"},{"key":"pone.0288670.ref011","doi-asserted-by":"crossref","first-page":"6528","DOI":"10.3390\/s21196528","article-title":"Using Smartwatches to Detect Face Touching","volume":"21","author":"C Bai","year":"2021","journal-title":"Sensors"},{"key":"pone.0288670.ref012","unstructured":"Bazarevsky V, Kartynnik Y, Vakunov A, Raveendran K, Grundmann M. BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs. arXiv; 2019. Available: http:\/\/arxiv.org\/abs\/1907.05047"},{"key":"pone.0288670.ref013","unstructured":"He Y, Xu D, Wu L, Jian M, Xiang S, Pan C. LFFD: A Light and Fast Face Detector for Edge Devices. arXiv; 2019. Available: http:\/\/arxiv.org\/abs\/1904.10633"},{"key":"pone.0288670.ref014","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"K Zhang","year":"2016","journal-title":"IEEE Signal Process Lett"},{"key":"pone.0288670.ref015","doi-asserted-by":"crossref","unstructured":"Deng J, Guo J, Zhou Y, Yu J, Kotsia I, Zafeiriou S. RetinaFace: Single-stage Dense Face Localisation in the Wild. arXiv; 2019. Available: http:\/\/arxiv.org\/abs\/1905.00641","DOI":"10.1109\/CVPR42600.2020.00525"},{"key":"pone.0288670.ref016","doi-asserted-by":"crossref","unstructured":"Hu P, Ramanan D. Finding Tiny Faces. arXiv; 2017. Available: http:\/\/arxiv.org\/abs\/1612.04402","DOI":"10.1109\/CVPR.2017.166"},{"key":"pone.0288670.ref017","first-page":"32","article-title":"Research on Face Intelligent Perception Technology Integrating Deep Learning under Different Illumination Intensities","volume":"1","author":"Y Yang","year":"2022","journal-title":"J Comput Cogn Eng"},{"key":"pone.0288670.ref018","doi-asserted-by":"crossref","unstructured":"Adiguna R, Soelistio YE. CNN Based Posture-Free Hand Detection. 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE). Kuta: IEEE; 2018. pp. 276\u2013279.","DOI":"10.1109\/ICITEED.2018.8534743"},{"key":"pone.0288670.ref019","author":"D Liu","year":"2020","journal-title":"Towards Interpretable and Robust Hand Detection via Pixel-wise Prediction"},{"key":"pone.0288670.ref020","doi-asserted-by":"crossref","unstructured":"Yang L, Qi Z, Liu Z, Zhou S, Zhang Y, Liu H, et al. A Light CNN based Method for Hand Detection and Orientation Estimation. 2018 24th International Conference on Pattern Recognition (ICPR). Beijing: IEEE; 2018. pp. 2050\u20132055.","DOI":"10.1109\/ICPR.2018.8545493"},{"key":"pone.0288670.ref021","doi-asserted-by":"crossref","first-page":"192","DOI":"10.3390\/s20010192","article-title":"Accurate Hand Detection from Single-Color Images by Reconstructing Hand Appearances","volume":"20","author":"C Xu","year":"2019","journal-title":"Sensors"},{"key":"pone.0288670.ref022","doi-asserted-by":"crossref","unstructured":"Kourbane I, Genc Y. Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation. arXiv; 2021. Available: http:\/\/arxiv.org\/abs\/2105.10904","DOI":"10.31449\/inf.v45i4.3470"},{"key":"pone.0288670.ref023","unstructured":"Mishra P, Sarawadekar K. Anchors Based Method for Fingertips Position Estimation from a Monocular RGB Image using Deep Neural Network. arXiv; 2020. Available: http:\/\/arxiv.org\/abs\/2005.01351"},{"key":"pone.0288670.ref024","doi-asserted-by":"crossref","unstructured":"Sen A, Mishra TK, Dash R. Design of Human Machine Interface through vision-based low-cost Hand Gesture Recognition system based on deep CNN. arXiv; 2022. Available: http:\/\/arxiv.org\/abs\/2207.03112","DOI":"10.21203\/rs.3.rs-1872480\/v1"},{"key":"pone.0288670.ref025","doi-asserted-by":"crossref","unstructured":"Xie H, Wang J, Shao B, Gu J, Li M. LE-HGR: A Lightweight and Efficient RGB-based Online Gesture Recognition Network for Embedded AR Devices. 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). 2019. pp. 274\u2013279.","DOI":"10.1109\/ISMAR-Adjunct.2019.00-30"},{"key":"pone.0288670.ref026","volume-title":"Supervised Contrastive Learning","author":"P Khosla","year":"2020"},{"key":"pone.0288670.ref027","doi-asserted-by":"crossref","unstructured":"Hoffer E, Ailon N. Deep metric learning using Triplet network. 2015. Available: http:\/\/arxiv.org\/abs\/1412.6622","DOI":"10.1007\/978-3-319-24261-3_7"},{"key":"pone.0288670.ref028","first-page":"207","article-title":"Distance Metric Learning for Large Margin Nearest Neighbor Classification","volume":"10","author":"KQ Weinberger","year":"2009","journal-title":"J Mach Learn Res"},{"key":"pone.0288670.ref029","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1007\/s11263-014-0733-5","article-title":"The Pascal Visual Object Classes Challenge: A Retrospective","volume":"111","author":"M Everingham","year":"2015","journal-title":"Int J Comput Vis"},{"key":"pone.0288670.ref030","doi-asserted-by":"crossref","unstructured":"Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2001. Kauai, HI, USA: IEEE Comput. Soc; 2001. p. I-511-I\u2013518.","DOI":"10.1109\/CVPR.2001.990517"},{"key":"pone.0288670.ref031","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE; 2016. pp. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0288670.ref032","unstructured":"Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv; 2017. Available: http:\/\/arxiv.org\/abs\/1704.04861"},{"key":"pone.0288670.ref033","doi-asserted-by":"crossref","unstructured":"Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C. MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE; 2018. pp. 4510\u20134520.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"pone.0288670.ref034","doi-asserted-by":"crossref","unstructured":"Ibrahim MR, Lyons T. ImageSig: A signature transform for ultra-lightweight image recognition. 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans, LA, USA: IEEE; 2022. pp. 3648\u20133658.","DOI":"10.1109\/CVPRW56347.2022.00409"},{"key":"pone.0288670.ref035","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv; 2021. Available: http:\/\/arxiv.org\/abs\/2010.11929"},{"key":"pone.0288670.ref036","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization. International Conference on Computer Vision (ICCV). 2017. p. 9.","DOI":"10.1109\/ICCV.2017.74"},{"key":"pone.0288670.ref037","doi-asserted-by":"crossref","first-page":"5202","DOI":"10.3390\/s20185202","article-title":"Helping the Blind to Get through COVID-19: Social Distancing Assistant Using Real-Time Semantic Segmentation on RGB-D Video","volume":"20","author":"M Martinez","year":"2020","journal-title":"Sensors"},{"key":"pone.0288670.ref038","unstructured":"Kreiss S, Bertoni L, Alahi A. OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association. arXiv; 2021. Available: http:\/\/arxiv.org\/abs\/2103.02440"},{"key":"pone.0288670.ref039","doi-asserted-by":"crossref","unstructured":"Kreiss S, Bertoni L, Alahi A. PifPaf: Composite Fields for Human Pose Estimation. arXiv; 2019. Available: http:\/\/arxiv.org\/abs\/1903.06593","DOI":"10.1109\/CVPR.2019.01225"},{"key":"pone.0288670.ref040","unstructured":"Cao Z, Hidalgo G, Simon T, Wei S-E, Sheikh Y. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. arXiv; 2019. Available: http:\/\/arxiv.org\/abs\/1812.08008"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0288670","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,21]],"date-time":"2024-11-21T20:20:19Z","timestamp":1732220419000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0288670"}},"subtitle":[],"editor":[{"given":"Nouman","family":"Ali","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,6,13]]},"references-count":40,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6,13]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0288670","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,13]]}}}