{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T07:08:37Z","timestamp":1750662517910,"version":"3.37.3"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T00:00:00Z","timestamp":1685664000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T00:00:00Z","timestamp":1685664000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program","doi-asserted-by":"crossref","award":["2018YFE0122900"],"award-info":[{"award-number":["2018YFE0122900"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61773224","62066033","61762069"],"award-info":[{"award-number":["61773224","62066033","61762069"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Applied Technology Research and Development Foundation of Inner Mongolia Autonomous Region","award":["2019GG372","2020GG0046","2021GG0158","2020PT0002","2021GG0165"],"award-info":[{"award-number":["2019GG372","2020GG0046","2021GG0158","2020PT0002","2021GG0165"]}]},{"name":"Achievements Transformation Project of Inner Mongolia Autonomous Region","award":["2019CG028"],"award-info":[{"award-number":["2019CG028"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Zero-shot sketch-based image retrieval (ZS-SBIR) is a challenging task that involves searching natural images related to a given hand-drawn sketch under the zero-shot scene. The previous approach projected image and sketch features into a low-dimensional common space for retrieval, and used semantic features to transfer the knowledge of seen to unseen classes. However, it is not effective enough to align multimodal features when projecting them into a common space, since the styles and contents of sketches and natural images are different and they are not one-to-one correspondence. To solve this problem, we propose a novel three-branch joint training network with contour detection network (called CDNNet) for the ZS-SBIR task, which uses contour maps as a bridge to align sketches and natural images to alleviate the domain gap. Specifically, we use semantic metrics to constrain the relationship between contour images and natural images and between contour images and sketches, so that natural image and sketch features can be aligned in the common space. Meanwhile, we further employ second-order attention to capture target subject information to increase the performance of retrieval descriptors. In addition, we use a teacher model and word embedding method to transfer the knowledge of the seen to the unseen classes. Extensive experiments on two large-scale datasets demonstrate that our proposed approach outperforms state-of-the-art CNN-based models: it improves by 2.6% on the Sketchy and 1.2% on TU-Berlin datasets in terms of mAP.<\/jats:p>","DOI":"10.1007\/s40747-023-01096-2","type":"journal-article","created":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T11:05:06Z","timestamp":1685703906000},"page":"6781-6795","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Contour detection network for zero-shot sketch-based image retrieval"],"prefix":"10.1007","volume":"9","author":[{"given":"Qing","family":"Zhang","sequence":"first","affiliation":[]},{"given":"Jing","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Xiangdong","family":"Su","sequence":"additional","affiliation":[]},{"given":"Feilong","family":"Bao","sequence":"additional","affiliation":[]},{"given":"Guanglai","family":"Gao","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,6,2]]},"reference":[{"key":"1096_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2022.108528","volume":"126","author":"O Tursun","year":"2022","unstructured":"Tursun O, Denman S, Sridharan S, Goan E, Fookes C (2022) An efficient framework for zero-shot sketch-based image retrieval. Pattern Recogn 126:108528","journal-title":"Pattern Recogn"},{"key":"1096_CR2","doi-asserted-by":"crossref","unstructured":"Tian J, Xu X, Shen F, Yang Y, Shen HT (2022) \u201cTVT: three-way vision transformer through multi-modal hypersphere learning for zero-shot sketch-based image retrieval,\u201d in Proceedings of the AAAI Conference on Artificial Intelligence 2370\u20132378","DOI":"10.1609\/aaai.v36i2.20136"},{"key":"1096_CR3","doi-asserted-by":"crossref","unstructured":"Bhunia AK et al. (2022) \u201cSketching without worrying: Noise-tolerant sketch-based image retrieval,\u201d in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 999\u20131008","DOI":"10.1109\/CVPR52688.2022.00107"},{"key":"1096_CR4","doi-asserted-by":"crossref","unstructured":"Guo L, Liu J, Wang Y, Luo Z, Wen W, Lu H (2017) \u201cSketch-based image retrieval using generative adversarial networks,\u201d in Proceedings of the 25th ACM international conference on Multimedia 1267\u20131268","DOI":"10.1145\/3123266.3127939"},{"key":"1096_CR5","doi-asserted-by":"publisher","first-page":"3737","DOI":"10.1109\/TIP.2022.3175403","volume":"31","author":"F Liu","year":"2022","unstructured":"Liu F et al (2022) SceneSketcher-v2: fine-grained scene-level sketch-based image retrieval using adaptive GCNs. IEEE Trans Image Process 31:3737\u20133751","journal-title":"IEEE Trans Image Process"},{"issue":"12","key":"1096_CR6","doi-asserted-by":"publisher","first-page":"9181","DOI":"10.1109\/TPAMI.2021.3123315","volume":"44","author":"H Wang","year":"2021","unstructured":"Wang H, Deng C, Liu T, Tao D (2021) Transferable coupled network for zero-shot sketch-based image retrieval. IEEE Trans Pattern Anal Mach Intell 44(12):9181\u20139194","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1096_CR7","doi-asserted-by":"crossref","unstructured":"Dey S, Riba P, Dutta A, Llados J, Song Y-Z (2019) \u201cDoodle to search: practical zero-shot sketch-based image retrieval,\u201d in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2179\u20132188","DOI":"10.1109\/CVPR.2019.00228"},{"key":"1096_CR8","doi-asserted-by":"crossref","unstructured":"Liu Q, Xie L, Wang H, Yuille AL (2019) \u201cSemantic-aware knowledge preservation for zero-shot sketch-based image retrieval,\u201d in Proceedings of the IEEE\/CVF International Conference on Computer Vision 3662\u20133671","DOI":"10.1109\/ICCV.2019.00376"},{"key":"1096_CR9","doi-asserted-by":"crossref","unstructured":"Wang Z, Wang H, Yan J, Wu A, Deng C (2021) \u201cDomain-smoothing network for zero-shot sketch-based image retrieval,\u201d In Proceedings of the International Joint Conference on Artificial Intelligence 1143\u20131149","DOI":"10.24963\/ijcai.2021\/158"},{"key":"1096_CR10","doi-asserted-by":"publisher","first-page":"8892","DOI":"10.1109\/TIP.2020.3020383","volume":"29","author":"C Deng","year":"2020","unstructured":"Deng C, Xu X, Wang H, Yang M, Tao D (2020) Progressive cross-modal semantic network for zero-shot sketch-based image retrieval. IEEE Trans Image Process 29:8892\u20138902","journal-title":"IEEE Trans Image Process"},{"key":"1096_CR11","doi-asserted-by":"crossref","unstructured":"Zhu J, Xu X, Shen F, Lee RK-W, Wang Z, Shen HT (2020) \u201cOcean: a dual learning approach for generalized zero-shot sketch-based image retrieval,\u201d in 2020 IEEE International Conference on Multimedia and Expo (ICME) 1\u20136","DOI":"10.1109\/ICME46284.2020.9102940"},{"key":"1096_CR12","doi-asserted-by":"crossref","unstructured":"Dutta A, Akata Z (2019) \u201cSemantically tied paired cycle consistency for zero-shot sketch-based image retrieval,\u201d in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 5089\u20135098","DOI":"10.1109\/CVPR.2019.00523"},{"key":"1096_CR13","doi-asserted-by":"crossref","unstructured":"Yelamarthi SK, Reddy SK, Mishra A, Mittal A (2018) \u201cA zero-shot framework for sketch based image retrieval,\u201d in Proceedings of the European Conference on Computer Vision (ECCV) 300\u2013317","DOI":"10.1007\/978-3-030-01225-0_19"},{"key":"1096_CR14","doi-asserted-by":"crossref","unstructured":"Lin K, Xu X, Gao L, Wang Z, Shen HT (2020) \u201cLearning cross-aligned latent embeddings for zero-shot cross-modal retrieval,\u201d in Proceedings of the AAAI Conference on Artificial Intelligence 11515\u201311522","DOI":"10.1609\/aaai.v34i07.6817"},{"key":"1096_CR15","doi-asserted-by":"crossref","unstructured":"Shen Y, Liu L, Shen F, Shao L (2018) \u201cZero-shot sketch-image hashing,\u201d in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3598\u20133607","DOI":"10.1109\/CVPR.2018.00379"},{"key":"1096_CR16","doi-asserted-by":"crossref","unstructured":"Yu Q, Liu F, Song Y-Z, Xiang T, Hospedales TM, Loy C-C (2016) \u201cSketch me that shoe,\u201d in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 799\u2013807","DOI":"10.1109\/CVPR.2016.93"},{"key":"1096_CR17","doi-asserted-by":"crossref","unstructured":"Lin H, Fu Y, Lu P, Gong S, Xue X, Jiang Y-G (2019) \u201cTc-net for isbir: Triplet classification network for instance-level sketch based image retrieval,\u201d in Proceedings of the 27th ACM international conference on multimedia 1676\u20131684","DOI":"10.1145\/3343031.3350900"},{"key":"1096_CR18","doi-asserted-by":"crossref","unstructured":"Ng T, Balntas V, Tian Y, Mikolajczyk K (2020) \u201cSOLAR: second-order loss and attention for image retrieval,\u201d in European Conference on Computer Vision 253\u2013270","DOI":"10.1007\/978-3-030-58595-2_16"},{"key":"1096_CR19","doi-asserted-by":"crossref","unstructured":"Lowe DG (1999) \u201cObject recognition from local scale-invariant features,\u201d in Proceedings of the seventh IEEE international conference on computer vision 2: 1150\u20131157","DOI":"10.1109\/ICCV.1999.790410"},{"issue":"7","key":"1096_CR20","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1016\/j.cviu.2013.02.005","volume":"117","author":"R Hu","year":"2013","unstructured":"Hu R, Collomosse J (2013) A performance evaluation of gradient field hog descriptor for sketch based image retrieval. Comput Vis Image Underst 117(7):790\u2013806","journal-title":"Comput Vis Image Underst"},{"key":"1096_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108291","volume":"122","author":"Y Chen","year":"2022","unstructured":"Chen Y et al (2022) AE-Net: fine-grained sketch-based image retrieval via attention-enhanced network. Pattern Recogn 122:108291","journal-title":"Pattern Recogn"},{"issue":"4","key":"1096_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2897824.2925954","volume":"35","author":"P Sangkloy","year":"2016","unstructured":"Sangkloy P, Burnell N, Ham C, Hays J (2016) The sketchy database: learning to retrieve badly drawn bunnies. ACM Trans Graph (TOG) 35(4):1\u201312","journal-title":"ACM Trans Graph (TOG)"},{"issue":"9","key":"1096_CR23","doi-asserted-by":"publisher","first-page":"2251","DOI":"10.1109\/TPAMI.2018.2857768","volume":"41","author":"Y Xian","year":"2018","unstructured":"Xian Y, Lampert CH, Schiele B, Akata Z (2018) Zero-shot learning\u2014a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell 41(9):2251\u20132265","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1096_CR24","doi-asserted-by":"crossref","unstructured":"Kodirov E, Xiang T, Gong S (2017) \u201cSemantic autoencoder for zero-shot learning,\u201d in Proceedings of the IEEE conference on computer vision and pattern recognition 3174\u20133183","DOI":"10.1109\/CVPR.2017.473"},{"key":"1096_CR25","unstructured":"Socher R, Ganjoo M, Manning CD, Ng A (2013) \u201cZero-shot learning through cross-modal transfer,\u201d Advances in neural information processing systems 26"},{"key":"1096_CR26","doi-asserted-by":"crossref","unstructured":"Changpinyo S, Chao W-L, Gong B, Sha F (2016) \u201cSynthesized classifiers for zero-shot learning,\u201d in Proceedings of the IEEE conference on computer vision and pattern recognition 5327\u20135336","DOI":"10.1109\/CVPR.2016.575"},{"key":"1096_CR27","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","volume":"6","author":"J Canny","year":"1986","unstructured":"Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 6:679\u2013698","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1096_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107657","volume":"110","author":"Q Zhang","year":"2021","unstructured":"Zhang Q, Lin C, Li F (2021) Application of binocular disparity and receptive field dynamics: a biologically-inspired model for contour detection. Pattern Recogn 110:107657","journal-title":"Pattern Recogn"},{"issue":"5","key":"1096_CR29","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1109\/TPAMI.2004.1273918","volume":"26","author":"DR Martin","year":"2004","unstructured":"Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530\u2013549","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"11","key":"1096_CR30","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6501\/ac8368","volume":"33","author":"H Tao","year":"2022","unstructured":"Tao H, Cheng L, Qiu J, Stojanovic V (2022) Few shot cross equipment fault diagnosis method based on parameter optimization and feature mertic. Meas Sci Technol 33(11):115005","journal-title":"Meas Sci Technol"},{"key":"1096_CR31","doi-asserted-by":"crossref","unstructured":"Shen L, Tao H, Ni Y, Wang Y, Vladimir S (2023) \u201cImproved YOLOv3 model with feature map cropping for multi-scale road object detection,\u201d Measurement Science and Technology","DOI":"10.1088\/1361-6501\/acb075"},{"key":"1096_CR32","doi-asserted-by":"crossref","unstructured":"Xie S, Tu Z (2015) \u201cHolistically-nested edge detection,\u201d in Proceedings of the IEEE international conference on computer vision 1395\u20131403","DOI":"10.1109\/ICCV.2015.164"},{"key":"1096_CR33","doi-asserted-by":"publisher","first-page":"761","DOI":"10.1109\/TMM.2020.2987685","volume":"23","author":"Y-J Cao","year":"2020","unstructured":"Cao Y-J, Lin C, Li Y-J (2020) Learning crisp boundaries using deep refinement network and adaptive weighting loss. IEEE Trans Multimedia 23:761\u2013771","journal-title":"IEEE Trans Multimedia"},{"key":"1096_CR34","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-022-03202-2","author":"C Lin","year":"2022","unstructured":"Lin C, Zhang Z, Hu Y (2022) Bio-inspired feature enhancement network for edge detection. Appl Intell. https:\/\/doi.org\/10.1007\/s10489-022-03202-2","journal-title":"Appl Intell"},{"key":"1096_CR35","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) \u201cDeep residual learning for image recognition,\u201d in Proceedings of the IEEE conference on computer vision and pattern recognition 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"1096_CR36","unstructured":"Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) \u201cDistributed representations of words and phrases and their compositionality,\u201d Advances in neural information processing systems 26"},{"key":"1096_CR37","doi-asserted-by":"crossref","unstructured":"Liu L, Shen F, Shen Y, Liu X, Shao L (2017) Deep sketch hashing: Fast free-hand sketch-based image retrieval,\u201d in Proceedings of the IEEE conference on computer vision and pattern recognition 2862\u20132871","DOI":"10.1109\/CVPR.2017.247"},{"issue":"12","key":"1096_CR38","doi-asserted-by":"publisher","first-page":"2916","DOI":"10.1109\/TPAMI.2012.193","volume":"35","author":"Y Gong","year":"2012","unstructured":"Gong Y, Lazebnik S, Gordo A, Perronnin F (2012) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35(12):2916\u20132929","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1096_CR39","unstructured":"Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11)"},{"key":"1096_CR40","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2022.02.079","author":"D Yang","year":"2022","unstructured":"Yang D, Peng B, Al-Huda Z, Malik A, Zhai D (2022) An overview of edge and object contour detection. Neurocomputing. https:\/\/doi.org\/10.1016\/j.neucom.2022.02.079","journal-title":"Neurocomputing"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01096-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01096-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01096-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,27]],"date-time":"2023-10-27T19:21:41Z","timestamp":1698434501000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01096-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,2]]},"references-count":40,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["1096"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01096-2","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"type":"print","value":"2199-4536"},{"type":"electronic","value":"2198-6053"}],"subject":[],"published":{"date-parts":[[2023,6,2]]},"assertion":[{"value":"10 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}