{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T05:40:36Z","timestamp":1777786836231,"version":"3.51.4"},"reference-count":67,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,9,27]]},"abstract":"<jats:p>The primary focus of this research is the discreet and subtle everyday contact interactions between mobile phones and their surrounding surfaces. Such interactions are anticipated to facilitate mobile context awareness, encompassing aspects such as dispensing medication updates, intelligently switching modes (e.g., silent mode), or initiating commands (e.g., deactivating an alarm). We introduce MicroCam, a contact-based sensing system that employs smartphone IMU data to detect the routine state of phone placement and utilizes a built-in microscope camera to capture intricate surface details. In particular, a natural dataset is collected to acquire authentic surface textures in situ for training and testing. Moreover, we optimize the deep neural network component of the algorithm, based on continual learning, to accurately discriminate between object categories (e.g., tables) and material constituents (e.g., wood). Experimental results highlight the superior accuracy, robustness and generalization of the proposed method. Lastly, we conducted a comprehensive discussion centered on our prototype, encompassing topics such as system performance and potential applications and scenarios.<\/jats:p>","DOI":"10.1145\/3610921","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T15:45:03Z","timestamp":1695829503000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["MicroCam"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1315-8969","authenticated-orcid":false,"given":"Yongquan","family":"Hu","sequence":"first","affiliation":[{"name":"University of New South Wales, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9512-828X","authenticated-orcid":false,"given":"Hui-Shyong","family":"Yeo","sequence":"additional","affiliation":[{"name":"Huawei, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5797-0945","authenticated-orcid":false,"given":"Mingyue","family":"Yuan","sequence":"additional","affiliation":[{"name":"University of New South Wales, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0551-1583","authenticated-orcid":false,"given":"Haoran","family":"Fan","sequence":"additional","affiliation":[{"name":"University of New South Wales, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8013-5989","authenticated-orcid":false,"given":"Don Samitha","family":"Elvitigala","sequence":"additional","affiliation":[{"name":"University of New South Wales, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4076-1811","authenticated-orcid":false,"given":"Wen","family":"Hu","sequence":"additional","affiliation":[{"name":"University of New South Wales, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5274-6889","authenticated-orcid":false,"given":"Aaron","family":"Quigley","sequence":"additional","affiliation":[{"name":"University of New South Wales, Australia"}]}],"member":"320","published-online":{"date-parts":[[2023,9,27]]},"reference":[{"key":"e_1_2_2_1_1","first-page":"445","article-title":"User-centered design","volume":"37","author":"Abras Chadia","year":"2004","unstructured":"Chadia Abras, Diane Maloney-Krichmar, Jenny Preece, et al. 2004. User-centered design. Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications 37, 4 (2004), 445--456.","journal-title":"Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/SYSMART.2016.7894491"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR48806.2021.9412614"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2835966.2835971"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1142405.1142446"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550316"},{"key":"e_1_2_2_7_1","unstructured":"Android Developers. 2021. BatteryHistorian. https:\/\/developer.android.com\/topic\/performance\/power\/setup-battery-historian Accessed: 2021-07-17."},{"key":"e_1_2_2_8_1","volume-title":"Understanding and using context. Personal and ubiquitous computing 5, 1","author":"Dey Anind K","year":"2001","unstructured":"Anind K Dey. 2001. Understanding and using context. Personal and ubiquitous computing 5, 1 (2001), 4--7."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-444-70304-0.50034-0"},{"key":"e_1_2_2_10_1","volume-title":"Proceedings of the 1st Annual Conference on Robot Learning (Proceedings of Machine Learning Research","volume":"166","author":"Erickson Zackory","unstructured":"Zackory Erickson, Sonia Chernova, and Charles C. Kemp. 2017. Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks. In Proceedings of the 1st Annual Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 78), Sergey Levine, Vincent Vanhoucke, and Ken Goldberg (Eds.). PMLR, 157--166. https:\/\/proceedings.mlr.press\/v78\/erickson17a.html"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2019.2892593"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341165"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3136815"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jdent.2020.103495"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCSW.2012.52"},{"key":"e_1_2_2_16_1","first-page":"5","article-title":"Human-centered vs. user-centered approaches to information system design","volume":"5","author":"Gasson Susan","year":"2003","unstructured":"Susan Gasson. 2003. Human-centered vs. user-centered approaches to information system design. Journal of Information Technology Theory and Application (JITTA) 5, 2 (2003), 5.","journal-title":"Journal of Information Technology Theory and Application (JITTA)"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1016587515822"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-92865-2_29"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2018.2870659"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449761"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994374.2994389"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550294"},{"key":"e_1_2_2_24_1","volume-title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861","author":"Howard Andrew G","year":"2017","unstructured":"Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2451176.2451206"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/3019350"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13673-019-0168-7"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494954"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380121"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132525.3134805"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376242"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2015.01.041"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534616"},{"key":"e_1_2_2_35_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024--8035. http:\/\/papers.neurips.cc\/paper\/9015- pytorch-an-imperative-style-high-performance-deep-learning-library.pdf"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025482"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSMC.2004.1400815"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300648"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12274-015-0969-6"},{"key":"e_1_2_2_40_1","volume-title":"From GUI to UUI: Interfaces for ubiquitous computing. Ubiquitous Computing Fundamentals","author":"Quigley Aaron","year":"2010","unstructured":"Aaron Quigley. 2010. From GUI to UUI: Interfaces for ubiquitous computing. Ubiquitous Computing Fundamentals (2010), 237--283."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/PLANS.2004.1309060"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/11426646_33"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2012.08.006"},{"key":"e_1_2_2_44_1","volume-title":"Experience replay for continual learning. Advances in Neural Information Processing Systems 32","author":"Rolnick David","year":"2019","unstructured":"David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702169"},{"key":"e_1_2_2_46_1","volume-title":"Implicit human computer interaction through context. Personal technologies 4, 2","author":"Schmidt Albrecht","year":"2000","unstructured":"Albrecht Schmidt. 2000. Implicit human computer interaction through context. Personal technologies 4, 2 (2000), 191--199."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411763.3451753"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300647"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448080"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3234695.3241015"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161416"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2019.1619259"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.3390\/app9081628"},{"key":"e_1_2_2_54_1","volume-title":"Resnet in resnet: Generalizing residual architectures. arXiv preprint arXiv:1603.08029","author":"Targ Sasha","year":"2016","unstructured":"Sasha Targ, Diogo Almeida, and Kevin Lyman. 2016. Resnet in resnet: Generalizing residual architectures. arXiv preprint arXiv:1603.08029 (2016)."},{"key":"e_1_2_2_55_1","unstructured":"Format Team. 2020. The Beginners Guide to Macro Photography. https:\/\/www.format.com\/magazine\/resources\/photography\/macro-photography-beginners-guide"},{"key":"e_1_2_2_56_1","volume-title":"BERT rediscovers the classical NLP pipeline. arXiv preprint arXiv:1905.05950","author":"Tenney Ian","year":"2019","unstructured":"Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. arXiv preprint arXiv:1905.05950 (2019)."},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1080\/00207590701396674"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376856"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1177\/016555158701300204"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2481296"},{"key":"e_1_2_2_61_1","volume-title":"Deep learning in microscopy image analysis: A survey","author":"Xing Fuyong","year":"2017","unstructured":"Fuyong Xing, Yuanpu Xie, Hai Su, Fujun Liu, and Lin Yang. 2017. Deep learning in microscopy image analysis: A survey. IEEE transactions on neural networks and learning systems 29, 10 (2017), 4550--4568."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3410530.3414613"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380137"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/UMEDIA.2008.4570872"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984515"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098279.3098541"},{"key":"e_1_2_2_67_1","volume-title":"International conference on machine learning. PMLR, 3987--3995","author":"Zenke Friedemann","year":"2017","unstructured":"Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In International conference on machine learning. PMLR, 3987--3995."}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610921","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610921","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,28]],"date-time":"2025-07-28T16:26:11Z","timestamp":1753719971000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610921"}},"subtitle":["Leveraging Smartphone Microscope Camera for Context-Aware Contact Surface Sensing"],"short-title":[],"issued":{"date-parts":[[2023,9,27]]},"references-count":67,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,9,27]]}},"alternative-id":["10.1145\/3610921"],"URL":"https:\/\/doi.org\/10.1145\/3610921","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,27]]},"assertion":[{"value":"2023-09-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}