{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T21:34:50Z","timestamp":1774992890225,"version":"3.50.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,9,27]]},"abstract":"<jats:p>Emerging terminals, such as smartwatches, true wireless earphones, in-vehicle computers, etc., are complementing our portals to ubiquitous information services. However, the current ecology of information services, encapsulated into millions of mobile apps, is largely restricted to smartphones; accommodating them to new devices requires tremendous and almost unbearable engineering efforts. Interaction Proxy, firstly proposed as an accessible technique, is a potential solution to mitigate this problem. Rather than re-building an entire application, Interaction Proxy constructs an alternative user interface that intercepts and translates interaction events and states between users and the original app's interface. However, in such a system, one key challenge is how to robustly and efficiently \"communicate\" with the original interface given the instability and dynamicity of mobile apps (e.g., dynamic application status and unstable layout). To handle this, we first define UI-Independent Application Description (UIAD), a reverse-engineered semantic model of mobile services, and then propose Interaction Proxy Manager (IPManager), which is responsible for synchronizing and managing the original apps' interface, and providing a concise programming interface that exposes information and method entries of the concerned mobile services. In this way, developers can build alternative interfaces without dealing with the complexity of communicating with the original app's interfaces. In this paper, we elaborate the design and implementation of our IPManager, and demonstrate its effectiveness by developing three typical proxies, mobile-smartwatch, mobile-vehicle and mobile-voice. We conclude by discussing the value of our approach to promote ubiquitous computing, as well as its limitations.<\/jats:p>","DOI":"10.1145\/3610929","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T15:45:03Z","timestamp":1695829503000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Interaction Proxy Manager"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-8639-5929","authenticated-orcid":false,"given":"Tian","family":"Huang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2591-7993","authenticated-orcid":false,"given":"Chun","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1351-9034","authenticated-orcid":false,"given":"Weinan","family":"Shi","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1358-722X","authenticated-orcid":false,"given":"Bowen","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8260-3830","authenticated-orcid":false,"given":"David","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6620-3420","authenticated-orcid":false,"given":"Yihao","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0003-5062","authenticated-orcid":false,"given":"Zhaoheng","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2273-6927","authenticated-orcid":false,"given":"Yuanchun","family":"Shi","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,27]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-021-00909-7"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/MODELS.2017.5"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2553035"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3459926.3464758"},{"key":"e_1_2_2_5_1","unstructured":"Lyan Alwakeel and Kevin Lano. 2020. Model Driven Development of Mobile Applications."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1654"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2046396.2046416"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066x.51.4.355"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3578522"},{"key":"e_1_2_2_11_1","volume-title":"Simulation and Code Generation for Web and Mobile Applications. In 2017 IEEE\/ACM 9th International Workshop on Modelling in Software Engineering (MiSE). 33--39","author":"Bernaschina Carlo","year":"2017","unstructured":"Carlo Bernaschina, Sara Comai, and Piero Fraternali. 2017. Online Model Editing, Simulation and Code Generation for Web and Mobile Applications. In 2017 IEEE\/ACM 9th International Workshop on Modelling in Software Engineering (MiSE). 33--39. https:\/\/doi.org\/10. 1109\/MiSE.2017.1"},{"key":"e_1_2_2_12_1","volume-title":"Proceedings of the 2013 Forum on specification and Design Languages (FDL). 1--8.","author":"Botturi G.","unstructured":"G. Botturi, E. Ebeid, F. Fummi, and D. Quaglia. 2013. Model-driven design for the development of multi-platform smartphone applications. In Proceedings of the 2013 Forum on specification and Design Languages (FDL). 1--8."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300792"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380327"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409691"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126651"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3185517"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207741"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604056"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490099.3511109"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357061"},{"key":"e_1_2_2_23_1","volume-title":"CAMELEON Project.","author":"Calvary D. Thevenin G.","year":"2002","unstructured":"D. Thevenin G. Calvary, J. Coutaz. 2002. The CAMELEON Reference Framework, CAMELEON Project. September 2002, available at http:\/\/giove.isti.cnr.it\/projects\/cameleon\/pdf\/cameleon%20d1.1refframework.pdf."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934595"},{"key":"e_1_2_2_25_1","first-page":"2","article-title":"Towards a better model based user interface development environment: A comprehensive survey","volume":"5","author":"Gomaa Mohammed","year":"2005","unstructured":"Mohammed Gomaa, Akram Salah, and Syed Rahman. 2005. Towards a better model based user interface development environment: A comprehensive survey. Proceedings of MICS 5 (2005), 2.","journal-title":"Proceedings of MICS"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-31753-3_38"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2449396.2449400"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3095300"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.06.095"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3483245"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3483245"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2015.7102582"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025483"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415820"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445049"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347899"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210240.3210339"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","unstructured":"Yang Li Jiacong He Xin Zhou Yuan Zhang and Jason Baldridge. 2020. mapping natural language instructions to mobile ui action sequences. (2020). https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.729","DOI":"10.18653\/v1\/2020.acl-main.729"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","unstructured":"Yang Li Gang Li Luheng He Jingjie Zheng Hong Li and Zhiwei Guan. 2020. widget captioning: generating natural language Description For Mobile User Interface Elements. (2020). https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-main.443","DOI":"10.18653\/v1\/2020.emnlp-main.443"},{"key":"e_1_2_2_40_1","volume-title":"ICWE Workshops. 325--338","author":"Limbourg Quentin","year":"2004","unstructured":"Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, and Murielle Florins. 2004. USIXML: A User Interface Description Language Supporting Multiple Levels of Independence.. In ICWE Workshops. 325--338."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242587.3242650"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1524\/icom.2011.0026"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180246"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.32"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517459"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10209-019-00650-5"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132525.3132558"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSRE.2019"},{"key":"e_1_2_2_49_1","unstructured":"Egbert Schlungbaum. 1996. Model-based user interface software tools-current state of declarative models. Technical Report. Georgia Institute of Technology."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2022.01307107"},{"key":"e_1_2_2_51_1","unstructured":"Hallvard Tr\u00e6tteberg. 2002. Model-based User Interface Design. Ph. D. Dissertation. http:\/\/hdl.handle.net\/11250\/249669"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/169059.169340"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397329"},{"key":"e_1_2_2_54_1","volume-title":"Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium","author":"Wallace Byron C.","unstructured":"Byron C. Wallace, Kevin Small, Carla E. Brodley, Joseph Lau, and Thomas A. Trikalinos. 2012. Deploying an Interactive Machine Learning System in an Evidence-Based Practice Center: Abstrackr. In Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium (Miami, Florida, USA) (IHI '12). Association for Computing Machinery, New York, NY, USA, 819--824. https:\/\/doi.org\/10. 1145\/2110363.2110464"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580895"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3472749.3474765"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3472749.3474763"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3117811.3117819"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","unstructured":"Seid Muhie Yimam Chris Biemann Ljiljana Majnaric \u0160efket \u0160abanovi\u0107 and Andreas Holzinger. 2015. Interactive and Iterative Annotation for Biomedical Entity Recognition. In Brain Informatics and Health Lecture Notes in Computer Science. 347--357. https:\/\/doi.org\/10.1007\/978-3-319-23344-4_34","DOI":"10.1007\/978-3-319-23344-4_34"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445186"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025846"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242587.3242616"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3234695.3236349"},{"key":"e_1_2_2_64_1","volume-title":"22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. https:\/\/doi.org\/10","author":"Zhou Zhilan","unstructured":"Zhilan Zhou, Jian Xu, Aruna Balasubramanian, and Donald E. Porter. 2020. A Survey of Patterns for Adapting Smartphone App UIs to Smart Watches. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. https:\/\/doi.org\/10. 1145\/3379503.3403564"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610929","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610929","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,28]],"date-time":"2025-07-28T16:27:02Z","timestamp":1753720022000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610929"}},"subtitle":["Semantic Model Generation and Run-time Support for Reconstructing Ubiquitous User Interfaces of Mobile Services"],"short-title":[],"issued":{"date-parts":[[2023,9,27]]},"references-count":64,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,9,27]]}},"alternative-id":["10.1145\/3610929"],"URL":"https:\/\/doi.org\/10.1145\/3610929","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,27]]},"assertion":[{"value":"2023-09-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}