{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:34:43Z","timestamp":1783438483121,"version":"3.54.6"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2020,6,15]],"date-time":"2020-06-15T00:00:00Z","timestamp":1592179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2020,6,15]]},"abstract":"<jats:p>In this paper, we present FingerTrak, a minimal-obtrusive wristband that enables continuous 3D finger tracking and hand pose estimation with four miniature thermal cameras mounted closely on a form-fitting wristband. FingerTrak explores the feasibility of continuously reconstructing the entire hand postures (20 finger joints positions) without the needs of seeing all fingers. We demonstrate that our system is able to estimate the entire hand posture by observing only the outline of the hand, i.e., hand silhouettes from the wrist using low-resolution (32 x 24) thermal cameras. A customized deep neural network is developed to learn to \"stitch\" these multi-view images and estimate 20 joints positions in 3D space. Our user study with 11 participants shows that the system can achieve an average angular error of 6.46\u00b0 when tested under the same background, and 8.06\u00b0 when tested under a different background. FingerTrak also shows encouraging results with the re-mounting of the device and has the potential to reconstruct some of the complicated poses. We conclude this paper with further discussions of the opportunities and challenges of this technology.<\/jats:p>","DOI":"10.1145\/3397306","type":"journal-article","created":{"date-parts":[[2020,6,15]],"date-time":"2020-06-15T22:30:37Z","timestamp":1592260237000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":73,"title":["FingerTrak"],"prefix":"10.1145","volume":"4","author":[{"given":"Fang","family":"Hu","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, New York, Shanghai Jiao Tong University, Shanghai"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"He","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York, Hangzhou Dianzi University, Hangzhou, Zhejiang"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Songlin","family":"Xu","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York, University of Science and Technology of China, Hefei, Anhui"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yin","family":"Li","sequence":"additional","affiliation":[{"name":"University of Wisconsin-Madison, 6730 Medical Science Center, Madison, Wisconsin"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,6,15]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208576"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-005-4881-5"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-005-6879-4"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_46"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139544"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.3390\/s16060811"},{"key":"e_1_2_2_8_1","first-page":"1273","article-title":"IMU sensor-based electronic goniometric glove for clinical finger movement analysis","volume":"18","author":"Connolly James","year":"2017","journal-title":"IEEE Sensors Journal"},{"key":"e_1_2_2_9_1","doi-asserted-by":"crossref","unstructured":"Martin de La Gorce David J Fleet and Nikos Paragios. 2011. Model-based 3d hand pose estimation from monocular video. IEEE transactions on pattern analysis and machine intelligence 33 9 (2011) 1793--1805.  Martin de La Gorce David J Fleet and Nikos Paragios. 2011. Model-based 3d hand pose estimation from monocular video. IEEE transactions on pattern analysis and machine intelligence 33 9 (2011) 1793--1805.","DOI":"10.1109\/TPAMI.2011.33"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647396"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5772\/50093"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2030112.2030154"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00878"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01109"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322957"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3305367.3327975"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00065"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2018.8448284"},{"key":"e_1_2_2_20_1","volume-title":"Breakthroughs in statistics","author":"Huber Peter J"},{"key":"e_1_2_2_21_1","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML.  Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML."},{"key":"e_1_2_2_22_1","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV). 118--134","author":"Iqbal Umar","year":"2018"},{"key":"e_1_2_2_23_1","volume-title":"11th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 14). 303--316.","author":"Kellogg Bryce"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098279.3098553"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380139"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2007.4373770"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2011.6094701"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267242.3267247"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130937"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2807972"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.3390\/s18051545"},{"key":"e_1_2_2_32_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Loshchilov Ilya","year":"2017"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025807"},{"key":"e_1_2_2_34_1","first-page":"3","article-title":"Efficient model-based 3D tracking of hand articulations using Kinect","volume":"1","author":"Oikonomidis Iason","year":"2011","journal-title":"BmVC"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126483"},{"key":"e_1_2_2_36_1","volume-title":"Multiwave: Complex Hand Gesture Recognition Using the Doppler Effect. In Graphics Interface. 97--106.","author":"Pittman Corey R","year":"2017"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.145"},{"key":"e_1_2_2_38_1","volume-title":"European Conference on Computer Vision. Springer, 356--371","author":"Rogez Gr\u00e9gory","year":"2014"},{"key":"e_1_2_2_39_1","doi-asserted-by":"crossref","unstructured":"Javier Romero Dimitrios Tzionas and Michael J. Black. 2017. Embodied Hands: Modeling and Capturing Hands and Bodies Together. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 36 6 (Nov. 2017).  Javier Romero Dimitrios Tzionas and Michael J. Black. 2017. Embodied Hands: Modeling and Capturing Hands and Bodies Together. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 36 6 (Nov. 2017).","DOI":"10.1145\/3130800.3130883"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2658779.2658787"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702179"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.494"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00017"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026005"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.735811"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.114"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2789168.2790129"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925965"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629500"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.1999.806928"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2699343.2699350"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347867"},{"key":"e_1_2_2_54_1","volume-title":"Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 437","author":"Zhang Cheng"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00779-018-1152-3"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807480"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984574"},{"key":"e_1_2_2_58_1","volume-title":"Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 471--471","author":"Zhao Yixin","year":"2015"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.525"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397306","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3397306","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:12:47Z","timestamp":1750201967000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397306"}},"subtitle":["Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist"],"short-title":[],"issued":{"date-parts":[[2020,6,15]]},"references-count":58,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,6,15]]}},"alternative-id":["10.1145\/3397306"],"URL":"https:\/\/doi.org\/10.1145\/3397306","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,15]]},"assertion":[{"value":"2020-06-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}