{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:21:24Z","timestamp":1750220484744,"version":"3.41.0"},"reference-count":82,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T00:00:00Z","timestamp":1643241600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,1,31]]},"abstract":"<jats:p>This article strives for a diversely supervised visual product search, where queries specify a diverse set of labels to search for. Where previous works have focused on representing attribute, instance, or category labels individually, we consider them together to create a diverse set of labels for visually describing products. We learn an embedding from the supervisory signal provided by every label to encode their interrelationships. Once trained, every label has a corresponding visual representation in the embedding space, which is an aggregation of selected items from the training set. At search time, composite query representations retrieve images that match a specific set of diverse labels. We form composite query representations by averaging over the aggregated representations of each diverse label in the specific set. For evaluation, we extend existing product datasets of cars and clothes with a diverse set of labels. Experiments show the benefits of our embedding for diversely supervised visual product search in seen and unseen product combinations and for discovering product design styles.<\/jats:p>","DOI":"10.1145\/3461646","type":"journal-article","created":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T19:44:21Z","timestamp":1643312661000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Diversely-Supervised Visual Product Search"],"prefix":"10.1145","volume":"18","author":[{"given":"William","family":"Thong","sequence":"first","affiliation":[{"name":"University of Amsterdam, Amsterdam, the Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cees G. M.","family":"Snoek","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, the Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,1,27]]},"reference":[{"key":"e_1_3_3_2_2","volume-title":"CVPR","author":"Ak Kenan E.","year":"2018","unstructured":"Kenan E. Ak, Ashraf A. Kassim, Joo Hwee Lim, and Jo Yew Tham. 2018. Learning attribute representations with localization for flexible fashion search. In CVPR."},{"key":"e_1_3_3_3_2","volume-title":"ICCV","author":"Ak Kenan E.","year":"2019","unstructured":"Kenan E. Ak, Joo Hwee Lim, Jo Yew Tham, and Ashraf A. Kassim. 2019. Attribute manipulation generative adversarial networks for fashion images. In ICCV."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2487986"},{"key":"e_1_3_3_5_2","volume-title":"ICCV","author":"Al-Halah Ziad","year":"2017","unstructured":"Ziad Al-Halah, Rainer Stiefelhagen, and Kristen Grauman. 2017. Fashion forward: Forecasting visual style in fashion. In ICCV."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766959"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/2986459.2986692"},{"key":"e_1_3_3_8_2","volume-title":"EMNLP","author":"Bowman Samuel R.","year":"2015","unstructured":"Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In EMNLP."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3375786"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/2984093.2984128"},{"key":"e_1_3_3_11_2","volume-title":"arXiv:2003.04297","author":"Chen Xinlei","year":"2020","unstructured":"Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020. Improved baselines with momentum contrastive learning. In arXiv:2003.04297."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.202"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995474"},{"key":"e_1_3_3_14_2","volume-title":"NAACL","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL."},{"key":"e_1_3_3_15_2","volume-title":"CVPR","author":"Farhadi Ali","year":"2009","unstructured":"Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In CVPR."},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.5555\/2981562.2981617"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.5555\/2976456.2976509"},{"key":"e_1_3_3_18_2","volume-title":"CVPR","author":"Gordo Albert","year":"2017","unstructured":"Albert Gordo and Diane Larlus. 2017. Beyond instance-level image retrieval: Leveraging captions to learn a global visual representation for semantic retrieval. In CVPR."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.100"},{"key":"e_1_3_3_20_2","volume-title":"ICCV","author":"Han Xintong","year":"2017","unstructured":"Xintong Han, Zuxuan Wu, Phoenix X. Huang, Xiao Zhang, Menglong Zhu, Yuan Li, Yang Zhao, and Larry S. Davis. 2017. Automatic spatially aware fashion concept discovery. In ICCV."},{"key":"e_1_3_3_21_2","volume-title":"CVPR","author":"He Kaiming","year":"2020","unstructured":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR."},{"key":"e_1_3_3_22_2","volume-title":"CVPR","author":"He Kaiming","year":"2016","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR."},{"key":"e_1_3_3_23_2","volume-title":"CVPR","author":"Hsiao Wei-Lin","year":"2018","unstructured":"Wei-Lin Hsiao and Kristen Grauman. 2018. Creating capsule wardrobes from fashion images. In CVPR."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.127"},{"issue":"1","key":"e_1_3_3_25_2","article-title":"Circle & search: Attribute-aware shoe retrieval","volume":"11","author":"Huang Junshi","year":"2014","unstructured":"Junshi Huang, Si Liu, Junliang Xing, Tao Mei, and Shuicheng Yan. 2014. Circle & search: Attribute-aware shoe retrieval. ACM Trans. Multim. Comput., Commun. Applic. 11, 1 (2014).","journal-title":"ACM Trans. Multim. Comput., Commun. Applic."},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2788621"},{"key":"e_1_3_3_27_2","volume-title":"CVPR","author":"Johnson Justin","year":"2015","unstructured":"Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. 2015. Image retrieval using scene graphs. In CVPR."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2598339"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.382"},{"key":"e_1_3_3_30_2","volume-title":"ECCV","author":"Kiapour M. Hadi","year":"2014","unstructured":"M. Hadi Kiapour, Kota Yamaguchi, Alexander C. Berg, and Tamara L. Berg. 2014. Hipster wars: Discovering elements of fashion styles. In ECCV."},{"key":"e_1_3_3_31_2","volume-title":"CVPR","author":"Kim Gunhee","year":"2015","unstructured":"Gunhee Kim, Seungwhan Moon, and Leonid Sigal. 2015. Ranking and retrieval of image sequences from multiple paragraph queries. In CVPR."},{"key":"e_1_3_3_32_2","volume-title":"ICLR","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/2354409.2354723"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.77"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.48"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.140"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_10"},{"key":"e_1_3_3_38_2","volume-title":"ECCV","author":"Lee Kuang-Huei","year":"2018","unstructured":"Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. 2018. Stacked cross attention for image-text matching. In ECCV."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2393347.2396471"},{"key":"e_1_3_3_40_2","volume-title":"CVPR","author":"Liu Weiyang","year":"2017","unstructured":"Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. Sphereface: Deep hypersphere embedding for face recognition. In CVPR."},{"key":"e_1_3_3_41_2","volume-title":"CVPR","author":"Liu Ziwei","year":"2016","unstructured":"Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. 2016. DeepFashion: Powering robust clothes recognition and retrieval with rich annotations. In CVPR."},{"key":"e_1_3_3_42_2","volume-title":"ICLR","author":"Loshchilov Ilya","year":"2017","unstructured":"Ilya Loshchilov and Frank Hutter. 2017. SGDR: Stochastic gradient descent with warm restarts. In ICLR."},{"key":"e_1_3_3_43_2","article-title":"Visualizing data using t-SNE","volume":"9","author":"van der Maaten Laurens","year":"2008","unstructured":"Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, Nov. (2008).","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_3_44_2","volume-title":"ICCV","author":"Mall Utkarsh","year":"2019","unstructured":"Utkarsh Mall, Kevin Matzen, Bharath Hariharan, Noah Snavely, and Kavita Bala. 2019. GeoStyle: Discovering fashion trends and events. In ICCV."},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.5555\/1394399"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767755"},{"key":"e_1_3_3_47_2","volume-title":"ICCV","author":"Movshovitz-Attias Yair","year":"2017","unstructured":"Yair Movshovitz-Attias, Alexander Toshev, Thomas K. Leung, Sergey Ioffe, and Saurabh Singh. 2017. No fuss distance metric learning using proxies. In ICCV."},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126281"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3455008"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.425"},{"key":"e_1_3_3_51_2","volume-title":"CVPR","author":"Ravi Hareesh","year":"2018","unstructured":"Hareesh Ravi, Lezi Wang, Carlos Muniz, Leonid Sigal, Dimitris Metaxas, and Mubbasir Kapadia. 2018. Show me a story: Towards coherent neural story illustration. In CVPR."},{"key":"e_1_3_3_52_2","volume-title":"EMNLP","author":"Reimers Nils","year":"2019","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In EMNLP."},{"key":"e_1_3_3_53_2","article-title":"Principles of categorization","author":"Rosch E. H.","year":"1978","unstructured":"E. H. Rosch. 1978. Principles of categorization. Cognition and Categorization, Lawrence Erlbaum (Ed.) (1978).","journal-title":"Cognition and Categorization"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014822"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_3_56_2","volume-title":"CVPR","author":"Sadeghi Mohammad Amin","year":"2011","unstructured":"Mohammad Amin Sadeghi and Ali Farhadi. 2011. Recognition using visual phrases. In CVPR."},{"key":"e_1_3_3_57_2","volume-title":"CVPR","author":"Scheirer Walter J.","year":"2012","unstructured":"Walter J. Scheirer, Neeraj Kumar, Peter N. Belhumeur, and Terrance E. Boult. 2012. Multi-attribute spaces: Calibration for attribute fusion and similarity search. In CVPR."},{"key":"e_1_3_3_58_2","volume-title":"CVPR","author":"Schroff Florian","year":"2015","unstructured":"Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A unified embedding for face recognition and clustering. In CVPR."},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995329"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/3294996.3295163"},{"key":"e_1_3_3_61_2","volume-title":"CVPR","author":"Song Hyun Oh","year":"2016","unstructured":"Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. 2016. Deep metric learning via lifted structured feature embedding. In CVPR."},{"issue":"4","key":"e_1_3_3_62_2","first-page":"116","article-title":"Analyzing fashion life cycles\u2014Principles and perspectives","volume":"45","author":"Sproles George B.","year":"1981","unstructured":"George B. Sproles. 1981. Analyzing fashion life cycles\u2014Principles and perspectives. J. Market. 45, 4 (1981), 116\u2013124.","journal-title":"J. Market."},{"key":"e_1_3_3_63_2","volume-title":"ICCV","author":"Strezoski Gjorgji","year":"2019","unstructured":"Gjorgji Strezoski, Nanne van Noord, and Marcel Worring. 2019. Many task learning with task routing. In ICCV."},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1108\/07363760610641127"},{"key":"e_1_3_3_65_2","volume-title":"CVPR","author":"Veit Andreas","year":"2017","unstructured":"Andreas Veit, Serge Belongie, and Theofanis Karaletsos. 2017. Conditional similarity networks. In CVPR."},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.527"},{"key":"e_1_3_3_67_2","volume-title":"CVPR","author":"Vo Nam","year":"2019","unstructured":"Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. 2019. Composing text and image for image retrieval\u2014An empirical odyssey. In CVPR."},{"key":"e_1_3_3_68_2","volume-title":"ECCV","author":"Wang Yu-Xiong","year":"2016","unstructured":"Yu-Xiong Wang and Martial Hebert. 2016. Learning to learn: Model regression networks for easy small sample learning. In ECCV."},{"key":"e_1_3_3_69_2","volume-title":"ICCV","author":"Wang Zihao","year":"2019","unstructured":"Zihao Wang, Xihui Liu, Hongsheng Li, Lu Sheng, Junjie Yan, Xiaogang Wang, and Jing Shao. 2019. Camp: Cross-modal adaptive message passing for text-image retrieval. In ICCV."},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1577078"},{"key":"e_1_3_3_71_2","volume-title":"NAACL","author":"Williams Adina","year":"2018","unstructured":"Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL."},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098162"},{"key":"e_1_3_3_73_2","volume-title":"CVPR","author":"Yang L.","year":"2015","unstructured":"L. Yang, P. Luo, C. C. Loy, and X. Tang. 2015. A large-scale car dataset for fine-grained categorization and verification. In CVPR."},{"key":"e_1_3_3_74_2","volume-title":"WACV","author":"Ye Linwei","year":"2018","unstructured":"Linwei Ye, Zhi Liu, and Yang Wang. 2018. Learning semantic segmentation with diverse supervision. In WACV."},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.32"},{"key":"e_1_3_3_76_2","article-title":"Improving the annotation of DeepFashion images for fine-grained attribute recognition","author":"Zakizadeh R.","year":"2018","unstructured":"R. Zakizadeh, M. Sasdelli, Y. Qian, and E. Vazquez. 2018. Improving the annotation of DeepFashion images for fine-grained attribute recognition. arXiv:1807.11674 (2018).","journal-title":"arXiv:1807.11674"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054201"},{"key":"e_1_3_3_78_2","volume-title":"BMVC","author":"Zhai Andrew","year":"2019","unstructured":"Andrew Zhai and Hao-Yu Wu. 2019. Classification is a strong baseline for deep metric learning. In BMVC."},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330739"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219820"},{"key":"e_1_3_3_81_2","volume-title":"CVPR","author":"Zhao Bo","year":"2017","unstructured":"Bo Zhao, Jiashi Feng, Xiao Wu, and Shuicheng Yan. 2017. Memory-augmented attribute manipulation networks for interactive fashion search. In CVPR."},{"key":"e_1_3_3_82_2","volume-title":"ECCV","author":"Zhao Xiangyun","year":"2018","unstructured":"Xiangyun Zhao, Haoxiang Li, Xiaohui Shen, Xiaodan Liang, and Ying Wu. 2018. A modulation module for multi-task learning with applications in image retrieval. In ECCV."},{"key":"e_1_3_3_83_2","volume-title":"ICCV","author":"Zhu Shizhan","year":"2017","unstructured":"Shizhan Zhu, Raquel Urtasun, Sanja Fidler, Dahua Lin, and Chen Change Loy. 2017. Be your own Prada: Fashion synthesis with structural coherence. In ICCV."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461646","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461646","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:49:04Z","timestamp":1750193344000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461646"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,27]]},"references-count":82,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1,31]]}},"alternative-id":["10.1145\/3461646"],"URL":"https:\/\/doi.org\/10.1145\/3461646","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2022,1,27]]},"assertion":[{"value":"2020-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}