{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T03:18:40Z","timestamp":1764645520070,"version":"3.41.0"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T00:00:00Z","timestamp":1599177600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000093","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["K25DK113242, U54EB020404"],"award-info":[{"award-number":["K25DK113242, U54EB020404"]}],"id":[{"id":"10.13039\/100000093","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Foundation","award":["CNS1915847, CNS1823201"],"award-info":[{"award-number":["CNS1915847, CNS1823201"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2020,9,4]]},"abstract":"<jats:p>The development and validation of computational models to detect daily human behaviors (e.g., eating, smoking, brushing) using wearable devices requires labeled data collected from the natural field environment, with tight time synchronization of the micro-behaviors (e.g., start\/end times of hand-to-mouth gestures during a smoking puff or an eating gesture) and the associated labels. Video data is increasingly being used for such label collection. Unfortunately, wearable devices and video cameras with independent (and drifting) clocks make tight time synchronization challenging. To address this issue, we present the Window Induced Shift Estimation method for Synchronization (SyncWISE) approach. We demonstrate the feasibility and effectiveness of our method by synchronizing the timestamps of a wearable camera and wearable accelerometer from 163 videos representing 45.2 hours of data from 21 participants enrolled in a real-world smoking cessation study. Our approach shows significant improvement over the state-of-the-art, even in the presence of high data loss, achieving 90% synchronization accuracy given a synchronization tolerance of 700 milliseconds. Our method also achieves state-of-the-art synchronization performance on the CMU-MMAC dataset.<\/jats:p>","DOI":"10.1145\/3411824","type":"journal-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T21:39:45Z","timestamp":1599255585000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["SyncWISE"],"prefix":"10.1145","volume":"4","author":[{"given":"Yun C.","family":"Zhang","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, School of Electrical and Computer Engineering and Center for Health Analytics and Informatics, Atlanta, Georgia, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shibo","family":"Zhang","sequence":"additional","affiliation":[{"name":"Northwestern University, Department of Preventive Medicine and Department of Computer Science, Chicago, Illinois, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miao","family":"Liu","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, School of Electrical and Computer Engineering and Center for Health Analytics and Informatics, Atlanta, Georgia, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elyse","family":"Daly","sequence":"additional","affiliation":[{"name":"Northwestern University, Department of Preventive Medicine, Chicago, Illinois, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Battalio","sequence":"additional","affiliation":[{"name":"Northwestern University, Department of Preventive Medicine, Chicago, Illinois, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Santosh","family":"Kumar","sequence":"additional","affiliation":[{"name":"University of Memphis, Department of Computer Science, Memphis, Tennessee, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bonnie","family":"Spring","sequence":"additional","affiliation":[{"name":"Northwestern University, Department of Preventive Medicine, Chicago, Illinois, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James M.","family":"Rehg","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, School of Interactive Computing and Center for Health Analytics and Informatics, Atlanta, Georgia, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nabil","family":"Alshurafa","sequence":"additional","affiliation":[{"name":"Northwestern University, Department of Preventive Medicine and Department of Computer Science, Chicago, Illinois, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,9,4]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314388"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027063.3053271"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3264900"},{"volume-title":"Counting Bites With Bits: Expert Workshop Addressing Calorie and Macronutrient Intake Monitoring. J Med Internet Res 21","year":"2019","author":"Alshurafa Nabil","key":"e_1_2_2_4_1"},{"volume-title":"2012 ACM\/IEEE 11th International Conference on Information Processing in Sensor Networks (IPSN). 269--280","year":"2012","author":"Amin Ahsan","key":"e_1_2_2_5_1"},{"volume-title":"Soundnet: Learning Sound Representations from Unlabeled Video. In Advances in neural information processing systems. 892--900.","year":"2016","author":"Aytar Yusuf","key":"e_1_2_2_6_1"},{"volume-title":"2015 Seventh International Conference on Ubiquitous and Future Networks. IEEE, 956--958","year":"2015","author":"Bae Jum-Han","key":"e_1_2_2_7_1"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04471-7_11"},{"volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3, Article 92 (Sept.","year":"2018","author":"Bi Shengjie","key":"e_1_2_2_9_1"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.2196\/12832"},{"volume-title":"Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04)","year":"2004","author":"Brugman Hennie","key":"e_1_2_2_11_1"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2221924.2221951"},{"volume-title":"Out of Time: Automated Lip Sync in the Wild. In Asian conference on computer vision. Springer, 251--263","year":"2016","author":"Chung Joon Son","key":"e_1_2_2_13_1"},{"volume-title":"Time Synchronization and Data Fusion for RGB-depth Cameras and Inertial Sensors in AAL Applications. In 2015 IEEE International Conference on Communication Workshop (ICCW). IEEE, 265--270","year":"2015","author":"Cippitelli Enea","key":"e_1_2_2_14_1"},{"key":"e_1_2_2_15_1","unstructured":"The SciPy community. 2019. SciPy Library Curve Fitting Method. https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.optimize.curve_fit.html. Accessed: 2020-05-15.  The SciPy community. 2019. SciPy Library Curve Fitting Method. https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.optimize.curve_fit.html. Accessed: 2020-05-15."},{"volume-title":"CHI 2009 Workshop. Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research.","year":"2009","author":"de la Torre Fernando","key":"e_1_2_2_16_1"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070942.2071027"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1979742.1979706"},{"volume-title":"Aretha Retrieved","year":"2020","author":"Franklin","key":"e_1_2_2_19_1"},{"volume-title":"Lensch","year":"2016","author":"Freeman Ido","key":"e_1_2_2_20_1"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2016.02.011"},{"volume-title":"Kiing Ing Wong, and Iain Murray","year":"2019","author":"Han Yi Chiew","key":"e_1_2_2_22_1"},{"key":"e_1_2_2_23_1","unstructured":"David Harwath Antonio Torralba and James Glass. 2016. Unsupervised Learning of Spoken Language with Visual Context. In Advances in Neural Information Processing Systems. 1858--1866.  David Harwath Antonio Torralba and James Glass. 2016. Unsupervised Learning of Spoken Language with Visual Context. In Advances in Neural Information Processing Systems. 1858--1866."},{"volume-title":"AMIA (American Medical Informatics Association) 2017 Annual Symposium. American Medical Informatics Association.","year":"2017","author":"Hnat Timothy","key":"e_1_2_2_24_1"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3131672.3131694"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2807526"},{"volume-title":"Havinga","year":"2019","author":"Kamminga Jacob W.","key":"e_1_2_2_27_1"},{"volume-title":"Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon). 242--245","year":"2010","author":"Sami","key":"e_1_2_2_28_1"},{"key":"e_1_2_2_29_1","doi-asserted-by":"crossref","unstructured":"Yin Li Miao Liu and James M Rehg. 2018. In the eye of beholder: Joint learning of gaze and actions in first person video. In ECCV.  Yin Li Miao Liu and James M Rehg. 2018. In the eye of beholder: Joint learning of gaze and actions in first person video. In ECCV.","DOI":"10.1007\/978-3-030-01228-1_38"},{"key":"e_1_2_2_30_1","unstructured":"Timecode Systems Limited. 2020. Timecode Systems. https:\/\/www.timecodesystems.com\/syncbac-pro\/. Accessed: 2020-05-15.  Timecode Systems Limited. 2020. Timecode Systems. https:\/\/www.timecodesystems.com\/syncbac-pro\/. Accessed: 2020-05-15."},{"key":"e_1_2_2_31_1","unstructured":"Miao Liu Xin Chen Yun Zhang Yin Li and James M Rehg. 2020. Attention Distillation for Learning Video Representations. In BMVC.  Miao Liu Xin Chen Yun Zhang Yin Li and James M Rehg. 2020. Attention Distillation for Learning Video Representations. In BMVC."},{"key":"e_1_2_2_32_1","unstructured":"MD2K. 2020. MotionSense - MD2K. https:\/\/md2k.org\/documentation\/data_dictionary\/raw_streams\/motionsense.html. Accessed: 2020-05-15.  MD2K. 2020. MotionSense - MD2K. https:\/\/md2k.org\/documentation\/data_dictionary\/raw_streams\/motionsense.html. Accessed: 2020-05-15."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/PLANS.2010.5507193"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2012.15"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351242"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2806897"},{"volume-title":"Video Matching. In ACM SIGGRAPH 2004 Papers","year":"2004","author":"Sand Peter","key":"e_1_2_2_37_1"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2004.1316761"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2011.2126569"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00931"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601208"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.3189\/2013JoG12J126"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2017.0-173"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.538"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397313"},{"volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 2, Article 88 (July","year":"2018","author":"Yun","key":"e_1_2_2_46_1"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3411824","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3411824","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3411824","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:48Z","timestamp":1750195488000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3411824"}},"subtitle":["Window Induced Shift Estimation for Synchronization of Video and Accelerometry from Wearable Sensors"],"short-title":[],"issued":{"date-parts":[[2020,9,4]]},"references-count":46,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,9,4]]}},"alternative-id":["10.1145\/3411824"],"URL":"https:\/\/doi.org\/10.1145\/3411824","relation":{},"ISSN":["2474-9567"],"issn-type":[{"type":"electronic","value":"2474-9567"}],"subject":[],"published":{"date-parts":[[2020,9,4]]},"assertion":[{"value":"2020-09-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}