{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T15:16:56Z","timestamp":1781623016948,"version":"3.54.5"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,4,27]],"date-time":"2022-04-27T00:00:00Z","timestamp":1651017600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,27]]},"DOI":"10.1145\/3491102.3502042","type":"proceedings-article","created":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T17:08:31Z","timestamp":1651165711000},"page":"1-13","source":"Crossref","is-referenced-by-count":30,"title":["Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale"],"prefix":"10.1145","author":[{"given":"Gang","family":"Li","sequence":"first","affiliation":[{"name":"Google Research, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gilles","family":"Baechler","sequence":"additional","affiliation":[{"name":"Google Research, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Tragut","sequence":"additional","affiliation":[{"name":"Google Research, Google, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Li","sequence":"additional","affiliation":[{"name":"Google Research, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,28]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"Chongyang Bai Xiaoxue Zang Ying Xu Srinivas Sunkara Abhinav Rastogi Jindong Chen and Blaise\u00a0Aguera y Arcas. 2021. UIBert: Learning Generic Multimodal Representations for UI Understanding. arxiv:2107.13731\u00a0[cs.CV]  Chongyang Bai Xiaoxue Zang Ying Xu Srinivas Sunkara Abhinav Rastogi Jindong Chen and Blaise\u00a0Aguera y Arcas. 2021. UIBert: Learning Generic Multimodal Representations for UI Understanding. arxiv:2107.13731\u00a0[cs.CV]"},{"key":"e_1_3_2_2_2_1","volume-title":"Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments. (April","author":"Burns Andrea","year":"2021","unstructured":"Andrea Burns , Deniz Arsan , Sanjna Agrawal , Ranjitha Kumar , Kate Saenko , and Bryan\u00a0 A Plummer . 2021. Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments. (April 2021 ). arxiv:2104.08560\u00a0[cs.CL] Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, and Bryan\u00a0A Plummer. 2021. Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments. (April 2021). arxiv:2104.08560\u00a0[cs.CL]"},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026044"},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409691"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824109"},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126651"},{"key":"e_1_3_2_2_8_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (Oct.","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (Oct. 2018). arxiv:1810.04805\u00a0[cs.CL] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (Oct. 2018). arxiv:1810.04805\u00a0[cs.CL]"},{"key":"e_1_3_2_2_9_1","unstructured":"Jingwen Fu Xiaoyi Zhang Yuwang Wang Wenjun Zeng Sam Yang and Grayson Hilliard. 2021. Understanding Mobile GUI: from Pixel-Words to Screen-Sentences. arXiv preprint arXiv:2105.11941(2021).  Jingwen Fu Xiaoyi Zhang Yuwang Wang Wenjun Zeng Sam Yang and Grayson Hilliard. 2021. Understanding Mobile GUI: from Pixel-Words to Screen-Sentences. arXiv preprint arXiv:2105.11941(2021)."},{"key":"e_1_3_2_2_10_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning -","volume":"70","author":"Gilmer Justin","year":"2017","unstructured":"Justin Gilmer , Samuel\u00a0 S. Schoenholz , Patrick\u00a0 F. Riley , Oriol Vinyals , and George\u00a0 E. Dahl . 2017 . Neural Message Passing for Quantum Chemistry . In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia) (ICML\u201917). JMLR.org, 1263\u20131272. Justin Gilmer, Samuel\u00a0S. Schoenholz, Patrick\u00a0F. Riley, Oriol Vinyals, and George\u00a0E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia) (ICML\u201917). JMLR.org, 1263\u20131272."},{"key":"e_1_3_2_2_11_1","unstructured":"Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arxiv:1512.03385\u00a0[cs.CV]  Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arxiv:1512.03385\u00a0[cs.CV]"},{"key":"e_1_3_2_2_12_1","volume-title":"Screen2Vec: Semantic Embedding of GUI Screens and GUI Components","author":"Jia-Jun Li Toby","unstructured":"Toby Jia-Jun Li , Lindsay Popowski , Tom Mitchell , and Brad\u00a0 A Myers . 2021. Screen2Vec: Semantic Embedding of GUI Screens and GUI Components . Association for Computing Machinery , New York, NY, USA . https:\/\/doi.org\/10.1145\/3411764.3445049 Toby Jia-Jun Li, Lindsay Popowski, Tom Mitchell, and Brad\u00a0A Myers. 2021. Screen2Vec: Semantic Embedding of GUI Screens and GUI Components. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3411764.3445049"},{"key":"e_1_3_2_2_13_1","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1\u201315","author":"Jia-Jun Li Toby","year":"2021","unstructured":"Toby Jia-Jun Li , Lindsay Popowski , Tom Mitchell , and Brad\u00a0 A Myers . 2021 . Screen2Vec: Semantic Embedding of GUI Screens and GUI Components . In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1\u201315 . Toby Jia-Jun Li, Lindsay Popowski, Tom Mitchell, and Brad\u00a0A Myers. 2021. Screen2Vec: Semantic Embedding of GUI Screens and GUI Components. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1\u201315."},{"key":"e_1_3_2_2_14_1","unstructured":"Yang Li Jiacong He Xin Zhou Yuan Zhang and Jason Baldridge. 2020. Mapping Natural Language Instructions to Mobile UI Action Sequences. arxiv:2005.03776\u00a0[cs.CL]  Yang Li Jiacong He Xin Zhou Yuan Zhang and Jason Baldridge. 2020. Mapping Natural Language Instructions to Mobile UI Action Sequences. arxiv:2005.03776\u00a0[cs.CL]"},{"key":"e_1_3_2_2_15_1","volume-title":"Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements.","author":"Li Yang","year":"2020","unstructured":"Yang Li , Gang Li , Luheng He , Jingjie Zheng , Hong Li , and Zhiwei Guan . 2020 . Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements. Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, and Zhiwei Guan. 2020. Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements."},{"key":"e_1_3_2_2_16_1","volume-title":"Learnable Fourier Features for Multi-dimensional Spatial Positional Encoding. In Thirty-Fifth Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=R0h3NUMao_U","author":"Li Yang","year":"2021","unstructured":"Yang Li , Si Si , Gang Li , Cho-Jui Hsieh , and Samy Bengio . 2021 . Learnable Fourier Features for Multi-dimensional Spatial Positional Encoding. In Thirty-Fifth Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=R0h3NUMao_U Yang Li, Si Si, Gang Li, Cho-Jui Hsieh, and Samy Bengio. 2021. Learnable Fourier Features for Multi-dimensional Spatial Positional Encoding. In Thirty-Fifth Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=R0h3NUMao_U"},{"key":"e_1_3_2_2_17_1","volume-title":"Learning Design Semantics for Mobile Apps. In The 31st Annual ACM Symposium on User Interface Software and Technology","author":"Liu F.","year":"2018","unstructured":"Thomas\u00a0 F. Liu , Mark Craft , Jason Situ , Ersin Yumer , Radomir Mech , and Ranjitha Kumar . 2018 . Learning Design Semantics for Mobile Apps. In The 31st Annual ACM Symposium on User Interface Software and Technology ( Berlin, Germany) (UIST \u201918). ACM, New York, NY, USA, 569\u2013579. https:\/\/doi.org\/10.1145\/3242587.3242650 Thomas\u00a0F. Liu, Mark Craft, Jason Situ, Ersin Yumer, Radomir Mech, and Ranjitha Kumar. 2018. Learning Design Semantics for Mobile Apps. In The 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST \u201918). ACM, New York, NY, USA, 569\u2013579. https:\/\/doi.org\/10.1145\/3242587.3242650"},{"key":"e_1_3_2_2_18_1","series-title":"Lecture Notes in Computer Science(2016), 21\u201337. https:\/\/doi.org\/10.1007\/978-3-319-46448-0_2","volume-title":"SSD: Single Shot MultiBox Detector","author":"Liu Wei","year":"2016","unstructured":"Wei Liu , Dragomir Anguelov , Dumitru Erhan , Christian Szegedy , Scott Reed , Cheng-Yang Fu , and Alexander\u00a0 C. Berg . 2016 . SSD: Single Shot MultiBox Detector . Lecture Notes in Computer Science(2016), 21\u201337. https:\/\/doi.org\/10.1007\/978-3-319-46448-0_2 Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander\u00a0C. Berg. 2016. SSD: Single Shot MultiBox Detector. Lecture Notes in Computer Science(2016), 21\u201337. https:\/\/doi.org\/10.1007\/978-3-319-46448-0_2"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2014.7025068"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.32"},{"key":"e_1_3_2_2_21_1","volume-title":"Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol.\u00a0139)","author":"Radford Alec","year":"2021","unstructured":"Alec Radford , Jong\u00a0Wook Kim , Chris Hallacy , Aditya Ramesh , Gabriel Goh , Sandhini Agarwal , Girish Sastry , Amanda Askell , Pamela Mishkin , Jack Clark , Gretchen Krueger , and Ilya Sutskever . 2021 . Learning Transferable Visual Models From Natural Language Supervision . In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol.\u00a0139) , Marina Meila and Tong Zhang (Eds.). PMLR, 8748\u20138763. https:\/\/proceedings.mlr.press\/v139\/radford21a.html Alec Radford, Jong\u00a0Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol.\u00a0139), Marina Meila and Tong Zhang (Eds.). PMLR, 8748\u20138763. https:\/\/proceedings.mlr.press\/v139\/radford21a.html"},{"key":"e_1_3_2_2_22_1","volume-title":"Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren , Kaiming He , Ross Girshick , and Jian Sun . 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 ( 2015 ), 91\u201399. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015), 91\u201399."},{"key":"e_1_3_2_2_23_1","volume-title":"A Review on Data Cleansing Methods for Big Data. Procedia Comput. Sci. 161 (Jan","author":"Ridzuan Fakhitah","year":"2019","unstructured":"Fakhitah Ridzuan and Wan Mohd\u00a0Nazmee Wan\u00a0Zainon . 2019. A Review on Data Cleansing Methods for Big Data. Procedia Comput. Sci. 161 (Jan . 2019 ), 731\u2013738. Fakhitah Ridzuan and Wan Mohd\u00a0Nazmee Wan\u00a0Zainon. 2019. A Review on Data Cleansing Methods for Big Data. Procedia Comput. Sci. 161 (Jan. 2019), 731\u2013738."},{"key":"e_1_3_2_2_24_1","volume-title":"UI Components Recognition System Based On Image Understanding. In 2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE, 65\u201371","author":"Sun Xiaolei","year":"2020","unstructured":"Xiaolei Sun , Tongyu Li , and Jianfeng Xu . 2020 . UI Components Recognition System Based On Image Understanding. In 2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE, 65\u201371 . Xiaolei Sun, Tongyu Li, and Jianfeng Xu. 2020. UI Components Recognition System Based On Image Understanding. In 2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE, 65\u201371."},{"key":"e_1_3_2_2_25_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998\u20136008.  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998\u20136008."},{"key":"e_1_3_2_2_26_1","volume-title":"Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning. (Aug","author":"Wang Bryan","year":"2021","unstructured":"Bryan Wang , Gang Li , Xin Zhou , Zhourong Chen , Tovi Grossman , and Yang Li. 2021. Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning. (Aug . 2021 ). arxiv:2108.03353\u00a0[cs.HC] Bryan Wang, Gang Li, Xin Zhou, Zhourong Chen, Tovi Grossman, and Yang Li. 2021. Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning. (Aug. 2021). arxiv:2108.03353\u00a0[cs.HC]"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661837"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3472749.3474763"},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463706"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"crossref","unstructured":"Xiaoxue Zang Ying Xu and Jindong Chen. 2021. Multimodal Icon Annotation For Mobile Applications. arXiv preprint arXiv:2107.04452(2021).  Xiaoxue Zang Ying Xu and Jindong Chen. 2021. Multimodal Icon Annotation For Mobile Applications. arXiv preprint arXiv:2107.04452(2021).","DOI":"10.1145\/3447526.3472064"},{"key":"e_1_3_2_2_31_1","volume-title":"Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels. (Jan.","author":"Zhang Xiaoyi","year":"2021","unstructured":"Xiaoyi Zhang , Lilian de Greef , Amanda Swearngin , Samuel White , Kyle Murray , Lisa Yu , Qi Shan , Jeffrey Nichols , Jason Wu , Chris Fleizach , Aaron Everitt , and Jeffrey\u00a0 P Bigham . 2021 . Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels. (Jan. 2021). arxiv:2101.04893\u00a0[cs.HC] Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, and Jeffrey\u00a0P Bigham. 2021. Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels. (Jan. 2021). arxiv:2101.04893\u00a0[cs.HC]"},{"key":"e_1_3_2_2_32_1","unstructured":"Xingyi Zhou Dequan Wang and Philipp Kr\u00e4henb\u00fchl. 2019. Objects as Points. arxiv:1904.07850\u00a0[cs.CV]  Xingyi Zhou Dequan Wang and Philipp Kr\u00e4henb\u00fchl. 2019. Objects as Points. arxiv:1904.07850\u00a0[cs.CV]"},{"key":"e_1_3_2_2_33_1","volume-title":"Deformable DETR: Deformable Transformers for End-to-End Object Detection. In International Conference on Learning Representations.","author":"Zhu Xizhou","year":"2021","unstructured":"Xizhou Zhu , Weijie Su , Lewei Lu , Bin Li , Xiaogang Wang , and Jifeng Dai . 2021 . Deformable DETR: Deformable Transformers for End-to-End Object Detection. In International Conference on Learning Representations. Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. 2021. Deformable DETR: Deformable Transformers for End-to-End Object Detection. In International Conference on Learning Representations."}],"event":{"name":"CHI '22: CHI Conference on Human Factors in Computing Systems","location":"New Orleans LA USA","acronym":"CHI '22","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491102.3502042","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3491102.3502042","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:21Z","timestamp":1750188681000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491102.3502042"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,27]]},"references-count":33,"alternative-id":["10.1145\/3491102.3502042","10.1145\/3491102"],"URL":"https:\/\/doi.org\/10.1145\/3491102.3502042","relation":{},"subject":[],"published":{"date-parts":[[2022,4,27]]}}}