{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T18:05:08Z","timestamp":1790877908007,"version":"4.1.0"},"reference-count":42,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T00:00:00Z","timestamp":1790812800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Relationship quality is typically assessed using self-report questionnaires, which are time-consuming and may not fully capture interpersonal processes during interaction. Automated facial expression analysis provides an opportunity to study nonverbal behavior during naturalistic interactions in a more objective and continuous manner. We characterize dyadic facial synchrony from video-based facial action unit time series and examine its association with marital relationship quality. Specifically, dyadic coordination is quantified using zero-lag cross-correlations between partners\u2019 facial behaviors across multiple action units. Computed over successive temporal windows, these synchrony measures form a multidimensional synchrony profile, termed the S-matrix. We evaluate the proposed method using an iMotions dataset comprising video recordings of in-person conversations, frame-level facial expression estimates, and Quality of Marriage Index (QMI) survey responses from 27 married couples. Results from an unsupervised clustering analysis demonstrate that the proposed synchrony representation provides better separation between couples with high and low marital relationship quality than conventional statistical feature baselines.<\/jats:p>","DOI":"10.1371\/journal.pone.0356071","type":"journal-article","created":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T17:32:16Z","timestamp":1790875936000},"page":"e0356071","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Characterizing multidimensional facial synchrony for relationship quality detection"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2028-8753","authenticated-orcid":true,"given":"Malak","family":"Fora","sequence":"first","affiliation":[{"name":"School of Systems Science and Industrial Engineering, Binghamton University","place":["New York, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alyssa","family":"Miville","sequence":"additional","affiliation":[{"name":"Binghamton University","place":["New York, United States of America"],"department":["Department of Psychology"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Mattson","sequence":"additional","affiliation":[{"name":"Binghamton University","place":["New York, United States of America"],"department":["Department of Psychology"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6410-1375","authenticated-orcid":true,"given":"Congyu","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Systems Science and Industrial Engineering, Binghamton University","place":["New York, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,10,1]]},"reference":[{"issue":"1","key":"pone.0356071.ref001","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1037\/emo0000799","article-title":"Faces synchronize when communication through spoken language is prevented","volume":"23","author":"F Zhao","year":"2023","journal-title":"Emotion"},{"key":"pone.0356071.ref002","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.3389\/fpsyg.2014.01323","article-title":"Nonverbal synchrony and affect in dyadic interactions","volume":"5","author":"W Tschacher","year":"2014","journal-title":"Front Psychol"},{"issue":"1","key":"pone.0356071.ref003","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s10919-019-00319-w","article-title":"Nonverbal Synchrony of Facial Movements and Expressions Predict Therapeutic Alliance During a Structured Psychotherapeutic Interview","volume":"44","author":"K Yokotani","year":"2019","journal-title":"J Nonverbal Behav"},{"key":"pone.0356071.ref004","volume-title":"Observation of facial affect communicated during dyadic interaction: comparing human versus computer coding","author":"AJ Miville","year":"2023"},{"issue":"3","key":"pone.0356071.ref005","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1111\/famp.12371","article-title":"Psychophysiological Synchrony During Verbal Interaction in Romantic Relationships","volume":"58","author":"J Coutinho","year":"2019","journal-title":"Fam Process"},{"key":"pone.0356071.ref006","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.copsyc.2014.12.021","article-title":"Social Baseline Theory: The Social Regulation of Risk and Effort","volume":"1","author":"JA Coan","year":"2015","journal-title":"Curr Opin Psychol"},{"issue":"4","key":"pone.0356071.ref007","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1177\/02654075221128259","article-title":"Accurately detecting happy facial expressions associates with perceptions of flirtatiousness","volume":"40","author":"ES Bibby","year":"2023","journal-title":"J Soc Pers Relatsh"},{"issue":"1","key":"pone.0356071.ref008","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1037\/0022-006X.69.1.125","article-title":"Marital adjustment and outcome following treatments for depression","volume":"69","author":"MA Whisman","year":"2001","journal-title":"J Consult Clin Psychol"},{"issue":"7","key":"pone.0356071.ref009","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1177\/02654075221138022","article-title":"The Role of Relationship Conflict for Momentary Loneliness and Affect in the Daily Lives of Older Couples","volume":"40","author":"E Weber","year":"2023","journal-title":"J Soc Pers Relat"},{"issue":"11","key":"pone.0356071.ref010","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1016\/j.socscimed.2007.02.007","article-title":"Health concordance within couples: a systematic review","volume":"64","author":"D Meyler","year":"2007","journal-title":"Soc Sci Med"},{"issue":"3","key":"pone.0356071.ref011","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1037\/a0019481","article-title":"The premarital communication roots of marital distress and divorce: the first five years of marriage","volume":"24","author":"HJ Markman","year":"2010","journal-title":"J Fam Psychol"},{"issue":"4","key":"pone.0356071.ref012","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1037\/a0017578","article-title":"Social support, problem solving, and the longitudinal course of newlywed marriage","volume":"98","author":"KT Sullivan","year":"2010","journal-title":"J Pers Soc Psychol"},{"issue":"1","key":"pone.0356071.ref013","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1037\/1040-3590.13.1.5","article-title":"Observation of couple conflicts: clinical assessment applications, stubborn truths, and shaky foundations","volume":"13","author":"RE Heyman","year":"2001","journal-title":"Psychol Assess"},{"issue":"1","key":"pone.0356071.ref014","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1207\/S15327698JFC0101_06","article-title":"Facial Expressions During Marital Conflict","volume":"1","author":"J Gottman","year":"2001","journal-title":"J Fam Commun"},{"issue":"6","key":"pone.0356071.ref015","doi-asserted-by":"crossref","first-page":"634","DOI":"10.1080\/13803395.2012.667069","article-title":"Emotion recognition and marital satisfaction in stroke","volume":"34","author":"LX Blonder","year":"2012","journal-title":"J Clin Exp Neuropsychol"},{"key":"pone.0356071.ref016","unstructured":"Van Boxtel A. Facial EMG as a tool for inferring affective states. In: Proceedings of measuring behavior. vol. 2010. 2010."},{"key":"pone.0356071.ref017","volume-title":"What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS)","author":"P Ekman","year":"1997"},{"key":"pone.0356071.ref018","doi-asserted-by":"crossref","unstructured":"Yang J, Zhang F, Chen B, Khan SU. Facial expression recognition based on facial action unit. In: 2019 tenth international green and sustainable computing conference (IGSC). IEEE; 2019. p. 1\u20136.","DOI":"10.1109\/IGSC48788.2019.8957163"},{"issue":"1","key":"pone.0356071.ref019","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1007\/s10044-012-0279-5","article-title":"Impact of action unit detection in automatic emotion recognition","volume":"17","author":"T Senechal","year":"2012","journal-title":"Pattern Anal Applic"},{"issue":"2","key":"pone.0356071.ref020","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1097\/NMD.0000000000001268","article-title":"Movement Synchrony and Facial Synchrony as Diagnostic Features of Depression: A Pilot Study","volume":"209","author":"U Altmann","year":"2021","journal-title":"J Nerv Ment Dis"},{"issue":"3","key":"pone.0356071.ref021","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1007\/BF02884957","article-title":"Measures of hostility as predictors of facial affect during social interaction: evidence for construct validity","volume":"20","author":"BH Brummett","year":"1998","journal-title":"Ann Behav Med"},{"issue":"1","key":"pone.0356071.ref022","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1038\/s42003-023-05461-2","article-title":"Synchronized affect in shared experiences strengthens social connection","volume":"6","author":"JH Cheong","year":"2023","journal-title":"Commun Biol"},{"key":"pone.0356071.ref023","volume-title":"Interactional synchrony in romantic couples: Linking dynamic systems of nonverbal behavior with outcome data","author":"DJ Garcia","year":"2021"},{"issue":"9","key":"pone.0356071.ref024","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.3390\/e24091307","article-title":"Cross-Correlation- and Entropy-Based Measures of Movement Synchrony: Non-Convergence of Measures Leads to Different Associations with Depressive Symptoms","volume":"24","author":"U Altmann","year":"2022","journal-title":"Entropy (Basel)"},{"key":"pone.0356071.ref025","unstructured":"Dale R, Fusaroli R, D\u00f8jbak H\u00e5konsson DD, Healey P, M\u00f8nster D, McGraw J. Beyond synchrony: Complementarity and asynchrony in joint action. In: Proceedings of the Annual Meeting of the Cognitive Science Society. vol. 35. 2013."},{"key":"pone.0356071.ref026","volume-title":"Measuring the Causal Dynamics of Facial Interaction with Convergent Cross Mapping","author":"EO Postma","year":"2016"},{"key":"pone.0356071.ref027","doi-asserted-by":"crossref","first-page":"1546303","DOI":"10.3389\/fpsyg.2025.1546303","article-title":"Emotional contagion in dyadic online video conferences-empirical evidence based on self-report and facial expression data","volume":"16","author":"AKG Marx","year":"2025","journal-title":"Front Psychol"},{"key":"pone.0356071.ref028","unstructured":"Fora M, Mattson R, Wu C. Individual and Marital Wellbeing Study. 2026. Available from: https:\/\/doi.org\/10.5281\/zenodo.18126449"},{"issue":"1","key":"pone.0356071.ref029","doi-asserted-by":"crossref","first-page":"141","DOI":"10.2307\/351302","article-title":"Measuring Marital Quality: A Critical Look at the Dependent Variable","volume":"45","author":"R Norton","year":"1983","journal-title":"J Marriage Fam"},{"issue":"1","key":"pone.0356071.ref030","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1037\/a0022441","article-title":"The reliability of relationship satisfaction: a reliability generalization meta-analysis","volume":"25","author":"JM Graham","year":"2011","journal-title":"J Fam Psychol"},{"key":"pone.0356071.ref031","unstructured":"iMotions. Facial Expression Analysis: The Definitive Guide. 2016. Available from: https:\/\/imotions.com\/facialexpression-guide-ebook\/"},{"key":"pone.0356071.ref032","doi-asserted-by":"crossref","first-page":"329","DOI":"10.3389\/fpsyg.2020.00329","article-title":"A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion","volume":"11","author":"L Kulke","year":"2020","journal-title":"Front Psychol"},{"key":"pone.0356071.ref033","doi-asserted-by":"crossref","unstructured":"El Kaliouby R, Robinson P. Real-time inference of complex mental states from facial expressions and head gestures. Real-time vision for human-computer interaction. 2005. p. 181\u2013200.","DOI":"10.1007\/0-387-27890-7_11"},{"issue":"4","key":"pone.0356071.ref034","doi-asserted-by":"crossref","first-page":"1446","DOI":"10.3758\/s13428-017-0996-1","article-title":"Facial expression analysis with AFFDEX and FACET: A validation study","volume":"50","author":"S St\u00f6ckli","year":"2018","journal-title":"Behav Res Methods"},{"issue":"4","key":"pone.0356071.ref035","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1177\/1088868311411164","article-title":"Temporal interpersonal emotion systems: the \u201cTIES\u201d that form relationships","volume":"15","author":"EA Butler","year":"2011","journal-title":"Pers Soc Psychol Rev"},{"key":"pone.0356071.ref036","unstructured":"Miville A, Fora M, Mattson R, Wu C. Statistical features of computer recognized facial affect during social support interactions predict married couples\u2019 relationship quality. 2024."},{"key":"pone.0356071.ref037","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.is.2015.04.007","article-title":"Time-series clustering \u2013 A decade review","volume":"53","author":"S Aghabozorgi","year":"2015","journal-title":"Information Systems"},{"issue":"2","key":"pone.0356071.ref038","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1111\/jmft.12002","article-title":"A Preliminary Investigation of a Telephone Administration of the Quality of Marriage Index","volume":"39","author":"SB Woods","year":"2012","journal-title":"J Marital Family Therapy"},{"issue":"1","key":"pone.0356071.ref039","first-page":"3","article-title":"The longitudinal course of marital quality and stability: a review of theory, method, and research","volume":"118","author":"BR Karney","year":"1995","journal-title":"Psychol Bull"},{"issue":"37","key":"pone.0356071.ref040","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.2106645118","article-title":"Eye contact marks the rise and fall of shared attention in conversation","volume":"118","author":"S Wohltjen","year":"2021","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pone.0356071.ref041","doi-asserted-by":"crossref","unstructured":"Tian Yl, Kanade T, Cohn JF. Eye-state action unit detection by gabor wavelets. In: International Conference on Multimodal Interfaces. Springer; 2000. p. 143\u2013150.","DOI":"10.1007\/3-540-40063-X_19"},{"key":"pone.0356071.ref042","first-page":"393","article-title":"Communicative functions of eye closing behaviours. In: Analysis of Verbal and Nonverbal Communication and Enactment","volume-title":"The Processing Issues: COST 2102 International Conference, Budapest, Hungary, September 7-10, 2010, Revised Selected Papers","author":"L Vincze","year":"2011"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0356071","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T17:32:25Z","timestamp":1790875945000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0356071"}},"subtitle":[],"editor":[{"given":"Sebastian","family":"Wallot","sequence":"first","affiliation":[{"name":"Leuphana University of L\u00fcneburg: Leuphana Universitat Luneburg","place":["GERMANY"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,10,1]]},"references-count":42,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,10,1]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0356071","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,1]]}}}