{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T23:06:45Z","timestamp":1768345605506,"version":"3.49.0"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2020,10,14]],"date-time":"2020-10-14T00:00:00Z","timestamp":1602633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"ANU-Data61 Collaborative Research Project","award":["CO19314"],"award-info":[{"award-number":["CO19314"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2020,10,14]]},"abstract":"<jats:p>Design sharing sites provide UI designers with a platform to share their works and also an opportunity to get inspiration from others' designs. To facilitate management and search of millions of UI design images, many design sharing sites adopt collaborative tagging systems by distributing the work of categorization to the community. However, designers often do not know how to properly tag one design image with compact textual description, resulting in unclear, incomplete, and inconsistent tags for uploaded examples which impede retrieval, according to our empirical study and interview with four professional designers. Based on a deep neural network, we introduce a novel approach for encoding both the visual and textual information to recover the missing tags for existing UI examples so that they can be more easily found by text queries. We achieve 82.72% accuracy in the tag prediction. Through a simulation test of 5 queries, our system on average returns hundreds more results than the default Dribbble search, leading to better relatedness, diversity and satisfaction.<\/jats:p>","DOI":"10.1145\/3415194","type":"journal-article","created":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T22:27:49Z","timestamp":1602800869000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["From Lost to Found"],"prefix":"10.1145","volume":"4","author":[{"given":"Chunyang","family":"Chen","sequence":"first","affiliation":[{"name":"Monash University, Melbourne, Australia"}]},{"given":"Sidong","family":"Feng","sequence":"additional","affiliation":[{"name":"Australian National University, Canberra, Australia"}]},{"given":"Zhengyang","family":"Liu","sequence":"additional","affiliation":[{"name":"Australian National University, Canberra, Australia"}]},{"given":"Zhenchang","family":"Xing","sequence":"additional","affiliation":[{"name":"Australian National University, Canberra, Australia"}]},{"given":"Shengdong","family":"Zhao","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore, Singapore"}]}],"member":"320","published-online":{"date-parts":[[2020,10,15]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"1998. Design Jobs and Portfolios | Coroflot. https:\/\/www.coroflot.com\/.  1998. Design Jobs and Portfolios | Coroflot. https:\/\/www.coroflot.com\/."},{"key":"e_1_2_1_2_1","unstructured":"2005. Automatic image annotation. https:\/\/en.wikipedia.org\/wiki\/Automatic_image_annotation.  2005. Automatic image annotation. https:\/\/en.wikipedia.org\/wiki\/Automatic_image_annotation."},{"key":"e_1_2_1_3_1","unstructured":"2005. Behance :: Best of Behance. https:\/\/www.behance.net\/.  2005. Behance :: Best of Behance. https:\/\/www.behance.net\/."},{"key":"e_1_2_1_4_1","unstructured":"2010. Dribbble - Discover the world's top designers & creatives. https:\/\/dribbble.com\/.  2010. Dribbble - Discover the world's top designers & creatives. https:\/\/dribbble.com\/."},{"key":"e_1_2_1_5_1","unstructured":"2014. Design - Material Design. https:\/\/material.io\/design\/.  2014. Design - Material Design. https:\/\/material.io\/design\/."},{"key":"e_1_2_1_6_1","unstructured":"2017. Essential Design Principles. https:\/\/developer.apple.com\/videos\/play\/wwdc2017\/802\/.  2017. Essential Design Principles. https:\/\/developer.apple.com\/videos\/play\/wwdc2017\/802\/."},{"key":"e_1_2_1_7_1","unstructured":"2017. Flickr: Find your inspiration. https:\/\/www.flickr.com\/.  2017. Flickr: Find your inspiration. https:\/\/www.flickr.com\/."},{"key":"e_1_2_1_8_1","unstructured":"2018. Dribbble 2018 Year in Review. https:\/\/dribbble.com\/year\/2018.  2018. Dribbble 2018 Year in Review. https:\/\/dribbble.com\/year\/2018."},{"key":"e_1_2_1_9_1","unstructured":"2020. Delicious. https:\/\/del.icio.us\/.  2020. Delicious. https:\/\/del.icio.us\/."},{"key":"e_1_2_1_10_1","unstructured":"2020. GloVe: Global Vectors for Word Representation. https:\/\/nlp.stanford.edu\/projects\/glove.  2020. GloVe: Global Vectors for Word Representation. https:\/\/nlp.stanford.edu\/projects\/glove."},{"key":"e_1_2_1_11_1","unstructured":"2020. Quora: A place to share knowledge and better understand the world. https:\/\/www.quora.com.  2020. Quora: A place to share knowledge and better understand the world. https:\/\/www.quora.com."},{"key":"e_1_2_1_12_1","volume-title":"Proc. 20th int. conf. very large data bases, VLDB","volume":"1215","author":"Agrawal Rakesh","year":"1994"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2785830.2785892"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v3i1.13937"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197231.3197244"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458112"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1088\/1742-5468\/2008\/10\/P10008"},{"key":"e_1_2_1_18_1","volume-title":"Understanding infographics through textual and visual tag prediction. arXiv preprint arXiv:1709.09215","author":"Bylinskii Zoya","year":"2017"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274302"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction 3, CSCW","author":"Chen Chunyang","year":"2019"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180240"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134667"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.48"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3391613","article-title":"Wireframe-based UI design search through image autoencoder","volume":"29","author":"Chen Jieshan","year":"2020","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409691"},{"key":"e_1_2_1_26_1","volume-title":"International conference on machine learning. 1274--1282","author":"Chen Minmin","year":"2013"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00070"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274307"},{"key":"e_1_2_1_29_1","volume-title":"Android User Interface Design: Implementing Material Design for Developers","author":"Clifton Ian G"},{"key":"e_1_2_1_30_1","volume-title":"Support-vector networks. Machine learning 20, 3","author":"Cortes Corinna","year":"1995"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126651"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2011.6116154"},{"key":"e_1_2_1_33_1","volume-title":"Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Statistics surveys 4","author":"Fay Michael P","year":"2010"},{"key":"e_1_2_1_34_1","unstructured":"Michael Fischer Richard R Yang and Monica S Lam. 2018. ImagineNet: Style Transfer from Fine Art to Graphical User Interfaces. (2018).  Michael Fischer Richard R Yang and Monica S Lam. 2018. ImagineNet: Style Transfer from Fine Art to Graphical User Interfaces. (2018)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1460563.1460600"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806923.1806926"},{"key":"e_1_2_1_37_1","volume-title":"The essential guide to user interface design: an introduction to GUI design principles and techniques","author":"Galitz Wilbert O"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/0165551506062337"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242602"},{"key":"e_1_2_1_40_1","volume-title":"Deep Residual Learning for Image Recognition. CoRR abs\/1512.03385","author":"He Kaiming","year":"2015"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027063.3053238"},{"key":"e_1_2_1_42_1","article-title":"Colors--Messengers of Concepts: Visual Design Mining for Learning Color Semantics","volume":"24","author":"Jahanian Ali","year":"2017","journal-title":"ACM Transactions on Computer-Human Interaction(TOCHI)"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/279044.279051"},{"key":"e_1_2_1_44_1","volume-title":"Wei Dong and Li Fei-fei","author":"Li Kai Li Jia Richard Socher","year":"2009"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.31"},{"key":"e_1_2_1_46_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014"},{"key":"e_1_2_1_47_1","unstructured":"Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.  Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_2_1_49_1","volume-title":"Soviet physics doklady","author":"Levenshtein Vladimir I"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675155"},{"key":"e_1_2_1_51_1","unstructured":"Lei Li Daqing He Danchen Zhang Yu Chi and Chengzhi Zhang. 2017. Types of Tags for Annotating Academic Blogs. (2017).  Lei Li Daqing He Danchen Zhang Yu Chi and Chengzhi Zhang. 2017. Types of Tags for Annotating Academic Blogs. (2017)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/WIIAT.2008.97"},{"key":"e_1_2_1_53_1","volume-title":"Synthetic Oversampling of Multi-Label Data based on Local Label Distribution. arXiv preprint arXiv:1905.00609","author":"Liu Bin","year":"2019"},{"key":"e_1_2_1_54_1","volume-title":"Learning Design Semantics for Mobile Apps. In The 31st Annual ACM Symposium on User Interface Software and Technology. ACM, 569--579","author":"Liu Thomas F","year":"2018"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 35th International Conference on Automated Software Engineering.","author":"Liu Zhe","year":"2020"},{"key":"e_1_2_1_56_1","volume-title":"Pacific Rim Conference on Multimedia. Springer, 25--35","author":"Ma Yuexin","year":"2017"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1108\/00242530610667558"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPCC.2007.4464070"},{"key":"e_1_2_1_59_1","unstructured":"Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.  Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1135777.1135961"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2671188.2749391"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/371920.371965"},{"key":"e_1_2_1_63_1","volume-title":"Mobile design pattern gallery: UI patterns for smartphone apps. \" O'Reilly Media","author":"Neil Theresa"},{"key":"e_1_2_1_64_1","volume-title":"Machine learning","author":"Quinlan J Ross"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2964284.2984068"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-017-0216-3"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1180875.1180904"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1367497.1367542"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/331403.331405"},{"key":"e_1_2_1_70_1","volume-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034","author":"Simonyan Karen","year":"2013"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458114"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174078"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/1363686.1364171"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-009-0362-0"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718497"},{"key":"e_1_2_1_76_1","volume-title":"Collaborative web tagging workshop at WWW2006","author":"Xu Zhichen"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380411"},{"key":"e_1_2_1_78_1","doi-asserted-by":"crossref","unstructured":"Shuyu Zheng Ziniu Hu and Yun Ma. 2019. FaceOff: Assisting the Manifestation Design of Web Graphical User Interface. (2019).  Shuyu Zheng Ziniu Hu and Yun Ma. 2019. FaceOff: Assisting the Manifestation Design of Web Graphical User Interface. (2019).","DOI":"10.1145\/3289600.3290610"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415194","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3415194","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:03:10Z","timestamp":1750197790000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415194"}},"subtitle":["Discover Missing UI Design Semantics through Recovering Missing Tags"],"short-title":[],"issued":{"date-parts":[[2020,10,14]]},"references-count":78,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2020,10,14]]}},"alternative-id":["10.1145\/3415194"],"URL":"https:\/\/doi.org\/10.1145\/3415194","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,14]]},"assertion":[{"value":"2020-10-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}