{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T15:33:33Z","timestamp":1774539213438,"version":"3.50.1"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,10,3]],"date-time":"2022-10-03T00:00:00Z","timestamp":1664755200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,3]],"date-time":"2022-10-03T00:00:00Z","timestamp":1664755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Discov Artif Intell"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The practice of using deep learning methods in safety critical vision systems such as autonomous driving has come a long way. As vision systems supported by deep learning methods become ubiquitous, the possible security threats faced by these systems have come into greater focus. As it is with any artificial intelligence system, these deep neural vision networks are first trained on a data set of interest, once they start performing well, they are deployed to a real-world environment. In the training stage, deep learning systems are susceptible to data poisoning attacks. While deep neural networks have proved to be versatile in solving a host of challenges. These systems have complex data ecosystems especially in computer vision. In practice, the security threats when training these systems are often ignored while deploying these models in the real world. However, these threats pose significant risks to the overall reliability of the system. In this paper, we present the fundamentals of data poisoning attacks when training deep learning vision systems and discuss countermeasures against these types of attacks. In addition, we simulate the risk posed by a real-world data poisoning attack on a deep learning vision system and present a novel algorithm MOVCE\u2014Model verification with Convolutional Neural Network and Word Embeddings which provides an effective countermeasure for maintaining the reliability of the system. The countermeasure described in this paper can be used on a wide variety of use cases where the risks posed by poisoning the training data are similar.<\/jats:p>","DOI":"10.1007\/s44163-022-00035-3","type":"journal-article","created":{"date-parts":[[2022,10,3]],"date-time":"2022-10-03T11:02:40Z","timestamp":1664794960000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["An improved real time detection of data poisoning attacks in deep learning vision systems"],"prefix":"10.1007","volume":"2","author":[{"given":"Vijay","family":"Raghavan","sequence":"first","affiliation":[]},{"given":"Thomas","family":"Mazzuchi","sequence":"additional","affiliation":[]},{"given":"Shahram","family":"Sarkani","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,10,3]]},"reference":[{"issue":"7553","key":"35_CR1","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436\u201344.","journal-title":"Nature"},{"key":"35_CR2","unstructured":"Tesla: Tesla transition. https:\/\/www.tesla.com\/support\/transitioning-tesla-vision. Accessed 20 Sept 2021."},{"key":"35_CR3","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/j.neucom.2020.10.081","volume":"429","author":"M Wang","year":"2021","unstructured":"Wang M, Deng W. Deep face recognition: a survey. Neurocomputing. 2021;429:215\u201344.","journal-title":"Neurocomputing"},{"key":"35_CR4","unstructured":"Dijk Tv, Croon Gd. How do neural networks see depth in single images? In: Proceedings of the IEEE\/CVF International conference on computer vision (ICCV). 2019."},{"key":"35_CR5","unstructured":"Kirkpatrick DD. DroneTarget. https:\/\/www.nytimes.com\/2020\/12\/02\/world\/middleeast\/iran-assassination-nuclear-scientist.html. Accessed 20 Sept 2021."},{"key":"35_CR6","doi-asserted-by":"publisher","unstructured":"Shi Y, Yu X, Sohn K, Chandraker M, Jain AK. Towards universal representation learning for deep face recognition. In: 2020 IEEE\/CVF conference on computer vision and pattern recognition (CVPR). 2020. p. 6816\u20136825. https:\/\/doi.org\/10.1109\/CVPR42600.2020.00685.","DOI":"10.1109\/CVPR42600.2020.00685"},{"issue":"1","key":"35_CR7","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1186\/s12916-020-01684-w","volume":"18","author":"KH Yu","year":"2020","unstructured":"Yu KH, Hu V, Wang F, Matulonis UA, Mutter GL, Golden JA, Kohane IS. Deciphering serous ovarian carcinoma histopathology and platinum response by convolutional neural networks. BMC Med. 2020;18(1):236. https:\/\/doi.org\/10.1186\/s12916-020-01684-w.","journal-title":"BMC Med"},{"key":"35_CR8","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1016\/j.ress.2018.11.011","volume":"182","author":"X Li","year":"2019","unstructured":"Li X, Zhang W, Ding Q. Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction. Reliab Eng Syst Saf. 2019;182:208\u201318.","journal-title":"Reliab Eng Syst Saf"},{"key":"35_CR9","doi-asserted-by":"publisher","first-page":"106901","DOI":"10.1016\/j.ress.2020.106901","volume":"199","author":"Z Xiang","year":"2020","unstructured":"Xiang Z, Bao Y, Tang Z, Li H. Deep reinforcement learning-based sampling method for structural reliability assessment. Reliab Eng Syst Saf. 2020;199:106901.","journal-title":"Reliab Eng Syst Saf"},{"key":"35_CR10","doi-asserted-by":"publisher","unstructured":"Sun C, Shrivastava A, Singh S, Gupta A. Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE International conference on computer vision. 2017. p. 843\u2013852. https:\/\/doi.org\/10.1109\/ICCV.2017.97.","DOI":"10.1109\/ICCV.2017.97"},{"key":"35_CR11","doi-asserted-by":"publisher","first-page":"74720","DOI":"10.1109\/ACCESS.2020.2987435","volume":"8","author":"M Xue","year":"2020","unstructured":"Xue M, Yuan C, Wu H, Zhang Y, Liu W. Machine learning security: threats, countermeasures, and evaluations. IEEE Access. 2020;8:74720\u201342. https:\/\/doi.org\/10.1109\/ACCESS.2020.2987435.","journal-title":"IEEE Access"},{"key":"35_CR12","unstructured":"Shafahi A, Huang WR, Najibi M, Suciu O, Studer C, Dumitras T, Goldstein T. Poison frogs! targeted clean-label poisoning attacks on neural networks. In: Proceedings of the 32nd International conference on neural information processing systems. 2018. p. 6106\u20136116."},{"key":"35_CR13","doi-asserted-by":"publisher","first-page":"106954","DOI":"10.1016\/j.ress.2020.106954","volume":"200","author":"G Islam","year":"2020","unstructured":"Islam G, Storer T. A case study of agile software development for safety-critical systems projects. Reliab Eng Syst Saf. 2020;200:106954.","journal-title":"Reliab Eng Syst Saf"},{"key":"35_CR14","doi-asserted-by":"publisher","first-page":"4566","DOI":"10.1109\/ACCESS.2020.3045078","volume":"9","author":"X Liu","year":"2021","unstructured":"Liu X, Xie L, Wang Y, Zou J, Xiong J, Ying Z, Vasilakos AV. Privacy and security issues in deep learning: a survey. IEEE Access. 2021;9:4566\u201393. https:\/\/doi.org\/10.1109\/ACCESS.2020.3045078.","journal-title":"IEEE Access"},{"key":"35_CR15","unstructured":"Sculley D. Hidden technical debt in Machine learning systems. In: Proceedings of the 28th International conference on neural information processing systems, vol. 2. Cambridge: MIT Press; 2015. p. 2503\u20132511."},{"key":"35_CR16","unstructured":"Schwarzschild A, Goldblum M, Gupta A, Dickerson JP, Goldstein T. Just how toxic is data poisoning? A unified benchmark for backdoor and data poisoning attacks. In: International conference on machine learning. PMLR; 2021. p. 9389\u20139398."},{"key":"35_CR17","doi-asserted-by":"publisher","unstructured":"Lewis GA, Ozkaya I, Xu X. Software architecture challenges for ml systems. In: 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). 2021. p. 634\u2013638. https:\/\/doi.org\/10.1109\/ICSME52107.2021.00071.","DOI":"10.1109\/ICSME52107.2021.00071"},{"key":"35_CR18","volume-title":"Data security for machine learning: data poisoning, backdoor attacks, and defenses","author":"M Goldblum","year":"2020","unstructured":"Goldblum M, Tsipras D, Xie C, Chen X, Schwarzschild A, Song D, Madry A, Li B, Goldstein T. Data security for machine learning: data poisoning, backdoor attacks, and defenses. New York: IEEE; 2020."},{"key":"35_CR19","unstructured":"Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng AY. Multimodal deep learning. In: ICML, 2011."},{"issue":"1","key":"35_CR20","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1007\/s10844-020-00608-7","volume":"56","author":"P Sawadogo","year":"2021","unstructured":"Sawadogo P, Darmont J. On data lake architectures and metadata management. J Intell Inform Syst. 2021;56(1):97\u2013120.","journal-title":"J Intell Inform Syst"},{"key":"35_CR21","doi-asserted-by":"publisher","DOI":"10.1201\/9781315108230","volume-title":"Feature engineering and selection: a practical approach for predictive models","author":"M Kuhn","year":"2019","unstructured":"Kuhn M, Johnson K. Feature engineering and selection: a practical approach for predictive models. Baco Raton: CRC Press; 2019."},{"issue":"5","key":"35_CR22","doi-asserted-by":"publisher","first-page":"829","DOI":"10.1162\/neco_a_01273","volume":"32","author":"J Gao","year":"2020","unstructured":"Gao J, Li P, Chen Z, Zhang J. A survey on deep learning for multimodal data fusion. Neural Comput. 2020;32(5):829\u201364.","journal-title":"Neural Comput"},{"issue":"6","key":"35_CR23","doi-asserted-by":"publisher","first-page":"1805","DOI":"10.1007\/s10618-020-00698-5","volume":"34","author":"VM Souza","year":"2020","unstructured":"Souza VM, dos Reis DM, Maletzke AG, Batista GE. Challenges in benchmarking stream learning algorithms with real-world data. Data Min Knowl Discov. 2020;34(6):1805\u201358.","journal-title":"Data Min Knowl Discov"},{"key":"35_CR24","doi-asserted-by":"publisher","unstructured":"Jagielski M, Oprea A, Biggio B, Liu C, Nita-Rotaru C, Li B. Manipulating machine learning: poisoning attacks and countermeasures for regression learning. In: 2018 IEEE symposium on security and privacy (SP). New York: IEEE; 2018. p. 19\u201335 https:\/\/doi.org\/10.1109\/SP.2018.00057.","DOI":"10.1109\/SP.2018.00057"},{"key":"35_CR25","unstructured":"Biggio B, Nelson B, Laskov P. Poisoning attacks against support vector machines. In: Proceedings of the 29th International conference on machine learning. 2012. p. 1467\u20131474."},{"key":"35_CR26","doi-asserted-by":"crossref","unstructured":"Aghakhani H, Meng D, Wang Y-X, Kruegel C, Vigna G. Bullseye polytope: a scalable clean-label poisoning attack with improved transferability. In: 2021 IEEE European symposium on security and privacy (EuroS&P). New York: IEEE; 2021. p. 159\u2013178.","DOI":"10.1109\/EuroSP51992.2021.00021"},{"key":"35_CR27","unstructured":"Zhu C, Huang WR, Li H, Taylor G, Studer C, Goldstein T. Transferable clean-label poisoning attacks on deep neural nets. In: International conference on machine learning. 2019. p. 7614\u20137623."},{"issue":"5","key":"35_CR28","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1111\/itor.12153","volume":"22","author":"JS Angelo","year":"2015","unstructured":"Angelo JS, Barbosa HJ. A study on the use of heuristics to solve a bilevel programming problem. Int Trans Oper Res. 2015;22(5):861\u201382.","journal-title":"Int Trans Oper Res"},{"key":"35_CR29","first-page":"12080","volume":"33","author":"WR Huang","year":"2020","unstructured":"Huang WR, Geiping J, Fowl L, Taylor G, Goldstein T. Metapoison: practical general-purpose clean-label data poisoning. Adv Neural Inf Process Syst. 2020;33:12080.","journal-title":"Adv Neural Inf Process Syst"},{"key":"35_CR30","volume-title":"Practical bilevel optimization: algorithms and applications","author":"JF Bard","year":"2013","unstructured":"Bard JF. Practical bilevel optimization: algorithms and applications, vol. 30. Raleigh: Springer; 2013."},{"issue":"1","key":"35_CR31","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1007\/s10479-007-0176-2","volume":"153","author":"B Colson","year":"2007","unstructured":"Colson B, Marcotte P, Savard G. An overview of bilevel optimization. Ann Oper Res. 2007;153(1):235\u201356. https:\/\/doi.org\/10.1007\/s10479-007-0176-2.","journal-title":"Ann Oper Res"},{"key":"35_CR32","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/j.eswa.2017.04.008","volume":"82","author":"TS Sethi","year":"2017","unstructured":"Sethi TS, Kantardzic M. On the reliable detection of concept drift from streaming unlabeled data. Expert Syst Appl. 2017;82:77\u201399. https:\/\/doi.org\/10.1016\/j.eswa.2017.04.008.","journal-title":"Expert Syst Appl"},{"key":"35_CR33","doi-asserted-by":"publisher","first-page":"546","DOI":"10.1016\/j.eswa.2017.10.003","volume":"92","author":"J Demsar","year":"2018","unstructured":"Demsar J, Bosnic Z. Detecting concept drift in data streams using model explanation. Expert Syst Appl. 2018;92:546\u201359. https:\/\/doi.org\/10.1016\/j.eswa.2017.10.003.","journal-title":"Expert Syst Appl"},{"key":"35_CR34","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1007\/3-540-44503","volume-title":"International conference on database theory","author":"P Buneman","year":"2001","unstructured":"Buneman P, Khanna S, Wang-Chiew T. Why and where: a characterization of data provenance. In: Van den Bussche J, Vianu V, editors. International conference on database theory. Heidelberg: Springer; 2001. p. 316\u201330. https:\/\/doi.org\/10.1007\/3-540-44503."},{"issue":"6","key":"35_CR35","doi-asserted-by":"publisher","first-page":"1279","DOI":"10.1109\/LCOMM.2019.2921755","volume":"24","author":"H Kim","year":"2019","unstructured":"Kim H, Park J, Bennis M, Kim SL. Blockchained on-device federated learning. IEEE Commun Lett. 2019;24(6):1279\u201383. https:\/\/doi.org\/10.1109\/LCOMM.2019.2921755.","journal-title":"IEEE Commun Lett"},{"key":"35_CR36","doi-asserted-by":"crossref","unstructured":"Barrak A, Eghan EE, Adams B. On the co-evolution of ml pipelines and source code-empirical study of dvc projects. In: 2021 IEEE International conference on software analysis, evolution and reengineering (SANER). New York: IEEE; 2021. p. 422\u2013433.","DOI":"10.1109\/SANER50967.2021.00046"},{"key":"35_CR37","doi-asserted-by":"crossref","unstructured":"Liu H, Zhu X, Lei Z, Li SZ. Adaptiveface: adaptive margin and sampling for face recognition. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR). 2019.","DOI":"10.1109\/CVPR.2019.01222"},{"key":"35_CR38","unstructured":"Krizhevsky A. Learning multiple layers of features from tiny images. Master\u2019s thesis, University of Tront. 2009."},{"key":"35_CR39","doi-asserted-by":"publisher","first-page":"106622","DOI":"10.1016\/j.knosys.2020.106622","volume":"212","author":"X He","year":"2021","unstructured":"He X, Zhao K, Chu X. Automl: a survey of the state-of-the-art. Knowl Based Syst. 2021;212:106622.","journal-title":"Knowl Based Syst"},{"key":"35_CR40","doi-asserted-by":"publisher","unstructured":"Truong A, Walters A, Goodsitt J, Hines K, Bruss CB, Farivar R. Towards automated machine learning: Evaluation and comparison of automl approaches and tools. In: 2019 IEEE 31st International conference on tools with artificial intelligence (ICTAI). 2019. p. 1471\u20131479. https:\/\/doi.org\/10.1109\/ICTAI.2019.00209.","DOI":"10.1109\/ICTAI.2019.00209"},{"key":"35_CR41","unstructured":"Le Q, Zoph B. Neural architecture search with reinforcement learning. 2016. https:\/\/arxiv.org\/abs\/1611.01578."},{"key":"35_CR42","doi-asserted-by":"publisher","DOI":"10.1145\/3065386","author":"A Krizhevsky","year":"2012","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst. 2012. https:\/\/doi.org\/10.1145\/3065386.","journal-title":"Adv Neural Inform Process Syst"},{"key":"35_CR43","unstructured":"LeCun Y, Huang FJ, Bottou L. Learning methods for generic object recognition with invariance to pose and lighting. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition. IEEE. 2004. p. 104."},{"key":"35_CR44","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1162\/tacl_a_00051","volume":"5","author":"P Bojanowski","year":"2017","unstructured":"Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching word vectors with subword information. Trans Assoc Comput Linguist. 2017;5:135\u201346.","journal-title":"Trans Assoc Comput Linguist"},{"issue":"4","key":"35_CR45","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1007\/s41019-021-00167-z","volume":"6","author":"R Sarki","year":"2021","unstructured":"Sarki R, Ahmed K, Wang H, Zhang Y, Ma J, Wang K. Image preprocessing in classification and identification of diabetic eye diseases. Data Sci Eng. 2021;6(4):455\u201371.","journal-title":"Data Sci Eng"}],"container-title":["Discover Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44163-022-00035-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44163-022-00035-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44163-022-00035-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,3]],"date-time":"2022-10-03T11:22:43Z","timestamp":1664796163000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44163-022-00035-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,3]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["35"],"URL":"https:\/\/doi.org\/10.1007\/s44163-022-00035-3","relation":{},"ISSN":["2731-0809"],"issn-type":[{"value":"2731-0809","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,3]]},"assertion":[{"value":"29 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"18"}}