{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T05:21:46Z","timestamp":1790832106238,"version":"4.1.0"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2025,12,2]]},"abstract":"<jats:p>Soft epidermal magnetic material has demonstrated excellent flexibility and wearability across a range of emerging applications, e.g., pressure sensing, robotics, among others. In this paper, we build a flexible magnetic skin based system MagSkin for touchless Human Computer Interaction (HCI). For example, a user can wear a Bandaid-like sticker on his fingertip to operate a smartphone using classic gestures without touching the screen, i.e., tap virtual icons floating on top of the screen. The magnetic skin is specially fabricated with randomly distributed magnetic microparticles, so that any shape change of the skin will cause position and orientation changes of those particles, introducing complex variations of the magnetic field around the skin. To characterize the magnetic field of the magnetic skin, we designed a multi-dipole based model. Based on this model, we developed 6-DoF position tracking and orientation estimation algorithms to localize the skin precisely. We also designed novel gestures, e.g., finger bend and pinch, for users to make input\/adjustment based on the flexibility of the material. We have conducted experiments with 15 volunteers for 18 hours, experimental results show localization accuracy is 6.2 mm in 3D, 4.5 mm in 2D, and recognition accuracy is 94% for classical gesture and 92% for novel gesture.<\/jats:p>","DOI":"10.1145\/3770675","type":"journal-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:42:32Z","timestamp":1764704552000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["MagSkin: Enabling Touchless Human-Computer Interaction with Soft Epidermal Magnetic Material"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-6744-901X","authenticated-orcid":false,"given":"Zeyu","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2817-2950","authenticated-orcid":false,"given":"Hua","family":"Huang","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of California, Merced, Merced, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8975-3911","authenticated-orcid":false,"given":"Penghao","family":"Dong","sequence":"additional","affiliation":[{"name":"Stony Brook University, Stony Brook, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2076-162X","authenticated-orcid":false,"given":"Shanshan","family":"Yao","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Stony Brook University, Stony Brook, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6362-2972","authenticated-orcid":false,"given":"Shan","family":"Lin","sequence":"additional","affiliation":[{"name":"Stony Brook University, Stony Brook, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2025. CONEXIO | KaVo Dental \u2014 kavo.com. https:\/\/www.kavo.com\/en\/products\/practice-equipment\/patient-communication\/conexio. [Accessed 29-07-2025]."},{"key":"e_1_2_1_2_1","unstructured":"2025. Wayfinding Kiosks | Parabit \u2014 parabit.com. https:\/\/www.parabit.com\/wayfinding-kiosks. [Accessed 29-07-2025]."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2851581.2892376"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/admt.201900493"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2460625.2460667"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.3003256"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449746"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3483260"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858125"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","unstructured":"Ananya Choudhury A. K. Talukdar and K. K. Sarma. 2015. A Review on Vision-Based Hand Gesture Recognition and Applications. (2015) 256\u2013281. doi:10.4018\/978-1-4666-8493-5.CH011","DOI":"10.4018\/978-1-4666-8493-5.CH011"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517512"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1039\/D3MH01062G"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00238-017-1337-4"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2022.3190389"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2015.2418456"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025808"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2022.3147904"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.201900025"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858095"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397306"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1021\/acsnano.2c08664"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307334.3326107"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994551.2994563"},{"key":"e_1_2_1_24_1","volume-title":"iBrush: Toothbrushing Monitoring using Smartwatch. ACM Transactions on Computing for Healthcare","author":"Huang Hua","year":"2025","unstructured":"Hua Huang and Shan Lin. 2025. iBrush: Toothbrushing Monitoring using Smartwatch. ACM Transactions on Computing for Healthcare (2025)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2016.53"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501988.2501991"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2021.3071287"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2022.10.014"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300568"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41378-021-00301-x"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/jsen.2021.3122236"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR55827.2022.00032"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478114"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466185"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925953"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2886279"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445543"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300638"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642052"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858580"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2024.103219"},{"key":"e_1_2_1_42_1","unstructured":"Nelson-Miller. [n. d.]. Zombie Finger: An Introduction to This Touchscreen Phenomenon. https:\/\/nelson-miller.com\/zombie-finger-an-introduction-to-this-touchscreen-phenomenon\/ Accessed: 2024-11-13."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21155098"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2014.2337331"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369831"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501980"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2015.08.004"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2500423.2500436"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compscitech.2022.109789"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","unstructured":"Elio Reinschmidt C. Vogt and M. Magno. 2022. Realtime Hand-Gesture Recognition Based on Novel Charge Variation Sensor and IMU. 2022 IEEE Sensors (2022) 1\u20134. doi:10.1109\/SENSORS52175.2022.9967346","DOI":"10.1109\/SENSORS52175.2022.9967346"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5624\/isd.2014.44.2.155"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-016-2525-z"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3241539.3241568"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2973750.2973764"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abc8801"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3141449"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-024-00904-9"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347867"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839462.2839463"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984529"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1687-0"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3090095"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1002\/adsr.202200008"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1002\/aelm.202300677"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2639108.2639110"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3770675","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:43:59Z","timestamp":1764704639000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3770675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":65,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,2]]}},"alternative-id":["10.1145\/3770675"],"URL":"https:\/\/doi.org\/10.1145\/3770675","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]},"assertion":[{"value":"2025-12-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}