{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T14:59:36Z","timestamp":1753887576987,"version":"3.41.2"},"reference-count":65,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T00:00:00Z","timestamp":1626739200000},"content-version":"vor","delay-in-days":200,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010909","name":"Excellent Young Scientists Fund","doi-asserted-by":"publisher","award":["61802050"],"award-info":[{"award-number":["61802050"]}],"id":[{"id":"10.13039\/501100010909","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U19A2059"],"award-info":[{"award-number":["U19A2059"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Wireless Communications and Mobile Computing"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Visual relationship can capture essential information for images, like the interactions between pairs of objects. Such relationships have become one prominent component of knowledge within sparse image data collected by multimedia sensing devices. Both the latent information and potential privacy can be included in the relationships. However, due to the high combinatorial complexity in modeling all potential relation triplets, previous studies on visual relationship detection have used the mixed visual and semantic features separately for each object, which is incapable for sparse data in IoT systems. Therefore, this paper proposes a new deep learning model for visual relationship detection, which is a novel attempt for cooperating computational intelligence (CI) methods with IoTs. The model imports the knowledge graph and adopts features for both entities and connections among them as extra information. It maps the visual features extracted from images into the knowledge\u2010based embedding vector space, so as to benefit from information in the background knowledge domain and alleviate the impacts of data sparsity. This is the first time that visual features are projected and combined with prior knowledge for visual relationship detection. Moreover, the complexity of the network is reduced by avoiding the learning of redundant features from images. Finally, we show the superiority of our model by evaluating on two datasets.<\/jats:p>","DOI":"10.1155\/2021\/6383646","type":"journal-article","created":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T23:20:09Z","timestamp":1626823209000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Robust Visual Relationship Detection towards Sparse Images in Internet\u2010of\u2010Things"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8740-876X","authenticated-orcid":false,"given":"Yang","family":"He","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6857-7744","authenticated-orcid":false,"given":"Guiduo","family":"Duan","sequence":"additional","affiliation":[]},{"given":"Guangchun","family":"Luo","sequence":"additional","affiliation":[]},{"given":"Xin","family":"Liu","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2021,7,20]]},"reference":[{"key":"e_1_2_9_1_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46448-0_51"},{"key":"e_1_2_9_2_2","doi-asserted-by":"crossref","unstructured":"GalleguillosC. RabinovichA. andBelongieS. Object categorization using co-occurrence location and appearance 2008 IEEE Conference on Computer Vision and Pattern Recognition June 2008 Anchorage AK USA 1\u20138 https:\/\/doi.org\/10.1109\/CVPR.2008.4587799 2-s2.0-51949110976.","DOI":"10.1109\/CVPR.2008.4587799"},{"key":"e_1_2_9_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"e_1_2_9_4_2","doi-asserted-by":"crossref","unstructured":"RedmonJ. DivvalaS. GirshickR. andFarhadiA. You only look once: unified real-time object detection 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2016 Las Vegas NV USA 779\u2013788 https:\/\/doi.org\/10.1109\/cvpr.2016.91 2-s2.0-84986308404.","DOI":"10.1109\/CVPR.2016.91"},{"key":"e_1_2_9_5_2","doi-asserted-by":"crossref","unstructured":"XiongZ. LiW. HanQ. andCaiZ. Privacy-preserving auto-driving: a GAN-based approach to protect vehicular camera data 2019 IEEE International Conference on Data Mining (ICDM) November 2019 Beijing China 668\u2013677 https:\/\/doi.org\/10.1109\/ICDM.2019.00077.","DOI":"10.1109\/ICDM.2019.00077"},{"key":"e_1_2_9_6_2","doi-asserted-by":"publisher","DOI":"10.26599\/TST.2019.9010029"},{"key":"e_1_2_9_7_2","doi-asserted-by":"publisher","DOI":"10.26599\/TST.2019.9010026"},{"key":"e_1_2_9_8_2","doi-asserted-by":"crossref","unstructured":"PlummerB. A. MallyaA. CervantesC. M. HockenmaierJ. andLazebnikS. Phrase localization and visual relationship detection with comprehensive image-language cues 2017 IEEE International Conference on Computer Vision (ICCV) October 2017 Venice Italy 1928\u20131937 https:\/\/doi.org\/10.1109\/iccv.2017.213 2-s2.0-85041911235.","DOI":"10.1109\/ICCV.2017.213"},{"key":"e_1_2_9_9_2","doi-asserted-by":"crossref","unstructured":"IzadiniaH. SadeghiF. andFarhadiA. Incorporating scene context and object layout into appearance modeling 2014 IEEE Conference on Computer Vision and Pattern Recognition June 2014 Columbus OH USA 232\u2013239 https:\/\/doi.org\/10.1109\/cvpr.2014.37 2-s2.0-84911457822.","DOI":"10.1109\/CVPR.2014.37"},{"key":"e_1_2_9_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2016.2643667"},{"key":"e_1_2_9_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2018.1700349"},{"key":"e_1_2_9_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0981-7"},{"key":"e_1_2_9_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2018.1701245"},{"key":"e_1_2_9_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2019.2911697"},{"key":"e_1_2_9_15_2","doi-asserted-by":"publisher","DOI":"10.26599\/TST.2021.9010026"},{"key":"e_1_2_9_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.3007662"},{"key":"e_1_2_9_17_2","doi-asserted-by":"crossref","unstructured":"SadeghiM. A.andFarhadiA. Recognition using visual phrases CVPR 2011 June 2011 Colorado Springs CO USA 1745\u20131752 https:\/\/doi.org\/10.1109\/CVPR.2011.5995711 2-s2.0-80052889458.","DOI":"10.1109\/CVPR.2011.5995711"},{"key":"e_1_2_9_18_2","doi-asserted-by":"crossref","unstructured":"LiY. OuyangW. WangX. andTangX. ViP-CNN: visual phrase guided convolutional neural network 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) July 2017 Honolulu HI USA 1347\u20131356 https:\/\/doi.org\/10.1109\/cvpr.2017.766 2-s2.0-85041906062.","DOI":"10.1109\/CVPR.2017.766"},{"key":"e_1_2_9_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnse.2018.2830307"},{"key":"e_1_2_9_20_2","doi-asserted-by":"crossref","unstructured":"CaiZ.andHeZ. Trading private range counting over big IoT data 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) July 2019 Dallas TX USA 144\u2013153 https:\/\/doi.org\/10.1109\/icdcs.2019.00023.","DOI":"10.1109\/ICDCS.2019.00023"},{"volume-title":"Generative adversarial networks: a survey towards private and secure applications","year":"2021","author":"Cai Z.","key":"e_1_2_9_21_2"},{"key":"e_1_2_9_22_2","unstructured":"AtzmonY. BerantJ. KezamiV. GlobersonA. andChechikG. Learning to generalize to new compositions in image understanding 2016 http:\/\/arxiv.org\/abs\/1608.07639."},{"key":"e_1_2_9_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15561-1_2"},{"key":"e_1_2_9_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-017-0490-5"},{"key":"e_1_2_9_25_2","doi-asserted-by":"crossref","unstructured":"DengJ. DongW. SocherR. LiL.-J. LiK. andFei-FeiL. ImageNet: a large-scale hierarchical image database 2009 IEEE conference on computer vision and pattern recognition June 2009 Miami FL USA 248\u2013255 https:\/\/doi.org\/10.1109\/cvpr.2009.5206848.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_9_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10590-1_4"},{"key":"e_1_2_9_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-009-0275-4"},{"key":"e_1_2_9_28_2","doi-asserted-by":"crossref","unstructured":"ZhuangB. LiuL. ShenC. andReidI. Towards context-aware interaction recognition for visual relationship detection 2017 IEEE International Conference on Computer Vision (ICCV) October 2017 Venice Italy 589\u2013598 https:\/\/doi.org\/10.1109\/iccv.2017.71 2-s2.0-85041911370.","DOI":"10.1109\/ICCV.2017.71"},{"key":"e_1_2_9_29_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33019185"},{"key":"e_1_2_9_30_2","unstructured":"MikolovT. ChenK. CorradoG. andDeanJ. Efficient estimation of word representations in vector space 2013 http:\/\/arxiv.org\/abs\/1301.3781."},{"key":"e_1_2_9_31_2","first-page":"2787","volume-title":"Advances in neural information processing systems","author":"Bordes A.","year":"2013"},{"key":"e_1_2_9_32_2","doi-asserted-by":"crossref","unstructured":"SchroffF. KalenichenkoD. andPhilbinJ. Facenet: a unified embedding for face recognition and clustering 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 815\u2013823 https:\/\/doi.org\/10.1109\/cvpr.2015.7298682 2-s2.0-84946751287.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_2_9_33_2","doi-asserted-by":"crossref","unstructured":"ChoiM. J. LimJ. J. TorralbaA. andWillskyA. S. Exploiting hierarchical context on a large database of object categories 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition June 2010 San Francisco CA USA 129\u2013136 https:\/\/doi.org\/10.1109\/cvpr.2010.5540221 2-s2.0-77956006912.","DOI":"10.1109\/CVPR.2010.5540221"},{"key":"e_1_2_9_34_2","doi-asserted-by":"crossref","unstructured":"KumarM. P.andKollerD. Efficiently selecting regions for scene understanding 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition June 2010 San Francisco CA USA 3217\u20133224 https:\/\/doi.org\/10.1109\/cvpr.2010.5540072 2-s2.0-77955997860.","DOI":"10.1109\/CVPR.2010.5540072"},{"key":"e_1_2_9_35_2","doi-asserted-by":"crossref","unstructured":"JohnsonJ. KrishnaR. StarkM. LiL.-J. ShammaD. BernsteinM. andFei-FeiL. Image retrieval using scene graphs 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 3668\u20133678 https:\/\/doi.org\/10.1109\/cvpr.2015.7298990 2-s2.0-84959233256.","DOI":"10.1109\/CVPR.2015.7298990"},{"key":"e_1_2_9_36_2","doi-asserted-by":"crossref","unstructured":"SchusterS. KrishnaR. ChangA. Fei-FeiL. andManningC. D. Generating semantically precise scene graphs from textual descriptions for improved image retrieval Proceedings of the Fourth Workshop on Vision and Language 2015 Lisbon Portugal 70\u201380 https:\/\/doi.org\/10.18653\/v1\/w15-2812.","DOI":"10.18653\/v1\/W15-2812"},{"key":"e_1_2_9_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.83"},{"key":"e_1_2_9_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-008-0140-x"},{"key":"e_1_2_9_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88682-2_3"},{"key":"e_1_2_9_40_2","doi-asserted-by":"crossref","unstructured":"YaoB.andFei-FeiL. Grouplet: a structured image representation for recognizing human and object interactions 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition June 2010 San Francisco CA USA 9\u201316 https:\/\/doi.org\/10.1109\/CVPR.2010.5540234 2-s2.0-77955987964.","DOI":"10.1109\/CVPR.2010.5540234"},{"key":"e_1_2_9_41_2","doi-asserted-by":"crossref","unstructured":"GkioxariG. GirshickR. andMalikJ. Contextual action recognition with R\u2217 CNN 2015 IEEE International Conference on Computer Vision (ICCV) December 2015 Santiago Chile 1080\u20131088 https:\/\/doi.org\/10.1109\/iccv.2015.129 2-s2.0-84973872492.","DOI":"10.1109\/ICCV.2015.129"},{"key":"e_1_2_9_42_2","doi-asserted-by":"crossref","unstructured":"RamanathanV. LiC. DengJ. HanW. LiZ. GuK. SongY. BengioS. RosenbergC. andFei-FeiL. Learning semantic relationships for better action retrieval in images 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 1100\u20131109 https:\/\/doi.org\/10.1109\/cvpr.2015.7298713 2-s2.0-84959233994.","DOI":"10.1109\/CVPR.2015.7298713"},{"key":"e_1_2_9_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2020.2980802"},{"key":"e_1_2_9_44_2","doi-asserted-by":"crossref","unstructured":"ZhaoH. PuigX. ZhouB. FidlerS. andTorralbaA. Open vocabulary scene parsing 2017 IEEE International Conference on Computer Vision (ICCV) October 2017 Venice Italy 2002\u20132010 https:\/\/doi.org\/10.1109\/iccv.2017.221 2-s2.0-85041927852.","DOI":"10.1109\/ICCV.2017.221"},{"key":"e_1_2_9_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-011-0439-x"},{"key":"e_1_2_9_46_2","doi-asserted-by":"crossref","unstructured":"SadeghiF. Kumar DivvalaS. K. andFarhadiA. Viske: visual knowledge extraction and question answering by visual verification of relation phrases 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 1456\u20131464 https:\/\/doi.org\/10.1109\/cvpr.2015.7298752 2-s2.0-84959184467.","DOI":"10.1109\/CVPR.2015.7298752"},{"key":"e_1_2_9_47_2","doi-asserted-by":"crossref","unstructured":"DaiB. ZhangY. andLinD. Detecting visual relationships with deep relational networks 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) July 2017 Honolulu HI USA 3076\u20133086 https:\/\/doi.org\/10.1109\/cvpr.2017.352 2-s2.0-85041892861.","DOI":"10.1109\/CVPR.2017.352"},{"key":"e_1_2_9_48_2","doi-asserted-by":"crossref","unstructured":"ZhangH. KyawZ. ChangS.-F. andChuaT.-S. Visual translation embedding network for visual relation detection 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) July 2017 Honolulu HI USA 5532\u20135540 https:\/\/doi.org\/10.1109\/cvpr.2017.331 2-s2.0-85029388674.","DOI":"10.1109\/CVPR.2017.331"},{"key":"e_1_2_9_49_2","doi-asserted-by":"crossref","unstructured":"DonahueJ. Anne HendricksL. GuadarramaS. RohrbachM. VenugopalanS. SaenkoK. andDarrellT. Long-term recurrent convolutional networks for visual recognition and description 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 2625\u20132634 https:\/\/doi.org\/10.1109\/cvpr.2015.7298878 2-s2.0-84959236502.","DOI":"10.1109\/CVPR.2015.7298878"},{"key":"e_1_2_9_50_2","doi-asserted-by":"crossref","unstructured":"KarpathyA.andFei-FeiL. Deep visual-semantic alignments for generating image descriptions 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 3128\u20133137 https:\/\/doi.org\/10.1109\/cvpr.2015.7298932 2-s2.0-84946734827.","DOI":"10.1109\/CVPR.2015.7298932"},{"key":"e_1_2_9_51_2","doi-asserted-by":"crossref","unstructured":"AntolS. AgrawalA. LuJ. MitchellM. BatraD. Lawrence ZitnickC. andParikhD. Vqa: visual question answering 2020 IEEE International Conference on Image Processing (ICIP) October 2015 Abu Dhabi UAE 2425\u20132433 https:\/\/doi.org\/10.1109\/icip40778.2020.9190828.","DOI":"10.1109\/ICIP40778.2020.9190828"},{"key":"e_1_2_9_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2754246"},{"key":"e_1_2_9_53_2","first-page":"2121","volume-title":"Advances in neural information processing systems","author":"Frome A.","year":"2013"},{"key":"e_1_2_9_54_2","unstructured":"VinyalsO. BlundellC. LillicrapT. WierstraD. andKavukcuogluK. Matching networks for one shot learning 30th Conference on Neural Information Processing Systems (NIPS 2016) 2016 Barcelona Spain 3630\u20133638."},{"key":"e_1_2_9_55_2","unstructured":"NorouziM. MikolovT. BengioS. SingerY. ShlensJ. FromeA. CorradoG. S. andDeanJ. Zero-shot learning by convex combination of semantic embeddings 2013 http:\/\/arxiv.org\/abs\/1312.5650."},{"key":"e_1_2_9_56_2","article-title":"Sherlock: scalable fact learning in images","volume":"31","author":"Elhoseiny M.","year":"2017","journal-title":"Thirty-First AAAI Conference on Artificial Intelligence"},{"key":"e_1_2_9_57_2","unstructured":"KirosR. SalakhutdinovR. andZemelR. S. Unifying visual-semantic embeddings with multimodal neural language models 2014 http:\/\/arxiv.org\/abs\/1411.2539."},{"key":"e_1_2_9_58_2","unstructured":"VendrovI. KirosR. FidlerS. andUrtasunR. Order-embeddings of images and language 2015 http:\/\/arxiv.org\/abs\/1511.06361."},{"key":"e_1_2_9_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0658-4"},{"key":"e_1_2_9_60_2","unstructured":"FaghriF. FleetD. J. KirosJ. R. andFidlerS. VSE++: improving visual-semantic embeddings with hard negatives 2017 http:\/\/arxiv.org\/abs\/1707.05612."},{"key":"e_1_2_9_61_2","first-page":"91","article-title":"Faster R-CNN: towards real-time object detection with region proposal networks","volume":"28","author":"Ren S.","year":"2015","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_9_62_2","unstructured":"SimonyanK.andZissermanA. Very deep convolutional networks for large-scale image recognition 2014 http:\/\/arxiv.org\/abs\/1409.1556."},{"key":"e_1_2_9_63_2","doi-asserted-by":"crossref","unstructured":"GirshickR. DonahueJ. DarrellT. andMalikJ. Rich feature hierarchies for accurate object detection and semantic segmentation 2014 IEEE Conference on Computer Vision and Pattern Recognition June 2014 Columbus OH USA 580\u2013587 https:\/\/doi.org\/10.1109\/cvpr.2014.81 2-s2.0-84911400494.","DOI":"10.1109\/CVPR.2014.81"},{"key":"e_1_2_9_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-013-5363-6"},{"key":"e_1_2_9_65_2","unstructured":"MikolovT. SutskeverI. ChenK. CorradoG. S. andDeanJ. Distributed representations of words and phrases and their compositionality Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 NIPS\u201913 Curran Associates Inc. 2013 Red Hook NY USA 3111\u20133119."}],"container-title":["Wireless Communications and Mobile Computing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2021\/6383646.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2021\/6383646.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2021\/6383646","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T12:25:53Z","timestamp":1723033553000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2021\/6383646"}},"subtitle":[],"editor":[{"given":"Yan","family":"Huang","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":65,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1155\/2021\/6383646"],"URL":"https:\/\/doi.org\/10.1155\/2021\/6383646","archive":["Portico"],"relation":{},"ISSN":["1530-8669","1530-8677"],"issn-type":[{"type":"print","value":"1530-8669"},{"type":"electronic","value":"1530-8677"}],"subject":[],"published":{"date-parts":[[2021,1]]},"assertion":[{"value":"2021-04-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"6383646"}}