{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T03:01:43Z","timestamp":1768532503249,"version":"3.49.0"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005005","name":"Ben-Gurion University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005005","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Object detection models, which are widely used in various domains (such as retail), have been shown to be vulnerable to adversarial attacks. Existing methods for detecting adversarial attacks on object detectors have had difficulty detecting new real-life attacks. We present X-Detect, a novel adversarial patch detector that can: (1) detect adversarial samples in real time, allowing the defender to take preventive action; (2) provide explanations for the alerts raised to support the defender\u2019s decision-making process, and (3) handle unfamiliar threats in the form of new attacks. Given a new scene, X-Detect uses an ensemble of explainable-by-design detectors that utilize object extraction, scene manipulation, and feature transformation techniques to determine whether an alert needs to be raised. X-Detect was evaluated in both the physical and digital space using five different attack scenarios (including adaptive attacks) and the benchmark COCO dataset and our new Superstore dataset. The physical evaluation was performed using a smart shopping cart setup in real-world settings and included 17 adversarial patch attacks recorded in 1700 adversarial videos. The results showed that X-Detect outperforms the state-of-the-art methods in distinguishing between benign and adversarial scenes for all attack scenarios while maintaining a 0% FPR (no false alarms) and providing actionable explanations for the alerts raised. A demo is available.<\/jats:p>","DOI":"10.1007\/s10994-024-06548-5","type":"journal-article","created":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T17:01:47Z","timestamp":1718816507000},"page":"6273-6292","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["X-Detect: explainable adversarial patch detection for object detectors in retail"],"prefix":"10.1007","volume":"113","author":[{"given":"Omer","family":"Hofman","sequence":"first","affiliation":[]},{"given":"Amit","family":"Giloni","sequence":"additional","affiliation":[]},{"given":"Yarin","family":"Hayun","sequence":"additional","affiliation":[]},{"given":"Ikuya","family":"Morikawa","sequence":"additional","affiliation":[]},{"given":"Toshiya","family":"Shimizu","sequence":"additional","affiliation":[]},{"given":"Yuval","family":"Elovici","sequence":"additional","affiliation":[]},{"given":"Asaf","family":"Shabtai","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,6,19]]},"reference":[{"key":"6548_CR1","doi-asserted-by":"crossref","unstructured":"Aldahdooh, A., Hamidouche, W., Fezza, S. A., & D\u00e9forges, O. (2022). Adversarial example detection for dnn models: A review and experimental comparison. Artificial Intelligence Review, 1\u201360.","DOI":"10.1007\/s10462-021-10125-w"},{"key":"6548_CR2","unstructured":"Amazon: Amazon shoplifting punishment detection. (2022). http:\/\/www.theverge.com\/2018\/1\/22\/16920784\/amazon-go-cashier-less-grocery-store-seattle-shoplifting-punishment-detection"},{"key":"6548_CR3","unstructured":"Brown, T. B., Man\u00e9, D., Roy, A., Abadi, M., & Gilmer, J. (2017). Adversarial patch. arXiv preprint arXiv:1712.09665"},{"key":"6548_CR5","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1609\/aaai.v35i2.16178","volume":"35","author":"Y Cai","year":"2021","unstructured":"Cai, Y., Wen, L., Zhang, L., Du, D., & Wang, W. (2021). Rethinking object detection in retail stores. Proceedings of the AAAI Conference on Artificial Intelligence, 35, 947\u2013954.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"5","key":"6548_CR4","doi-asserted-by":"publisher","first-page":"1483","DOI":"10.1109\/TPAMI.2019.2956516","volume":"43","author":"Z Cai","year":"2019","unstructured":"Cai, Z., & Vasconcelos, N. (2019). Cascade r-cnn: High quality object detection and instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5), 1483\u20131498.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"6548_CR7","unstructured":"Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Madry, A., & Kurakin, A. (2019). On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705"},{"key":"6548_CR6","doi-asserted-by":"crossref","unstructured":"Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (sp) (pp. 39\u201357). IEEE.","DOI":"10.1109\/SP.2017.49"},{"issue":"1","key":"6548_CR8","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1049\/cit2.12028","volume":"6","author":"A Chakraborty","year":"2021","unstructured":"Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., & Mukhopadhyay, D. (2021). A survey on adversarial attacks and defences. CAAI Transactions on Intelligence Technology, 6(1), 25\u201345.","journal-title":"CAAI Transactions on Intelligence Technology"},{"key":"6548_CR9","unstructured":"Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z., Cheng, D., Zhu, C., Cheng, T., Zhao, Q., Li, B., Lu, X., Zhu, R., Wu, Y., Dai, J., Wang, J., Shi, J., Ouyang, W., Loy, C. C., & Lin, D. (2019). MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155"},{"issue":"2","key":"6548_CR10","doi-asserted-by":"publisher","first-page":"71","DOI":"10.32604\/jbd.2020.012294","volume":"2","author":"K Chen","year":"2020","unstructured":"Chen, K., Zhu, H., Yan, L., & Wang, J. (2020). A survey on adversarial examples in deep learning. Journal on Big Data, 2(2), 71.","journal-title":"Journal on Big Data"},{"key":"6548_CR11","doi-asserted-by":"crossref","unstructured":"Chiang, P.-H., Chan, C.-S., & Wu, S.-H. (2021). Adversarial pixel masking: A defense against physical attacks for pre-trained object detectors. In Proceedings of the 29th ACM international conference on multimedia (pp. 1856\u20131865).","DOI":"10.1145\/3474085.3475338"},{"key":"6548_CR12","doi-asserted-by":"crossref","unstructured":"Chou, E., Tramer, F., & Pellegrino, G. (2020). Sentinet: Detecting localized universal attacks against deep learning systems. In 2020 IEEE security and privacy workshops (SPW) (pp. 48\u201354). IEEE.","DOI":"10.1109\/SPW50608.2020.00025"},{"key":"6548_CR13","unstructured":"Everingham, M., Van\u00a0Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2012). The PASCAL visual object classes challenge 2012 (VOC2012) results. http:\/\/www.pascal-network.org\/challenges\/VOC\/voc2012\/workshop\/index.html"},{"key":"6548_CR14","unstructured":"Federation, N. R. (2022). National Retail Security Survey. https:\/\/nrf.com\/research\/national-retail-security-survey-2022"},{"key":"6548_CR15","unstructured":"Forbes: Forbes Shoplifting Report. (2022). https:\/\/forbes.com\/sites\/jiawertz\/2022\/11\/20\/shoplifting-has-become-a-100-billion-problem-for-retailers\/?sh=679b9a282d62"},{"key":"6548_CR16","doi-asserted-by":"crossref","unstructured":"Fuchs, K., Grundmann, T., & Fleisch, E. (2019). Towards identification of packaged products via computer vision: Convolutional neural networks for object detection and image classification in retail environments. In Proceedings of the 9th international conference on the Internet of Things (pp. 1\u20138).","DOI":"10.1145\/3365871.3365899"},{"key":"6548_CR17","unstructured":"Green, K. M. (2021). Super-big market-data: A case study, walkthrough approach to amazon go cashierless convenience stores. PhD thesis, University of Illinois at Chicago."},{"key":"6548_CR18","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., & Sun, J.(2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770\u2013778).","DOI":"10.1109\/CVPR.2016.90"},{"key":"6548_CR19","doi-asserted-by":"crossref","unstructured":"Hu, Y.-C.-T., Kung, B.-H., Tan, D. S., Chen, J.-C., Hua, K.-L., & Cheng, W.-H. (2021). Naturalistic physical adversarial patch for object detectors. In Proceedings of the IEEE\/CVF international conference on computer vision (pp. 7848\u20137857).","DOI":"10.1109\/ICCV48922.2021.00775"},{"key":"6548_CR20","unstructured":"Ji, N., Feng, Y., Xie, H., Xiang, X., & Liu, N. (2021). Adversarial yolo: Defense human detection patch attacks via detecting adversarial patches. arXiv preprint arXiv:2103.08860"},{"issue":"11","key":"6548_CR21","doi-asserted-by":"publisher","first-page":"3365","DOI":"10.1109\/TVCG.2019.2921336","volume":"26","author":"Y Jing","year":"2019","unstructured":"Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., & Song, M. (2019). Neural style transfer: A review. IEEE Transactions on Visualization and Computer Graphics, 26(11), 3365\u20133385.","journal-title":"IEEE Transactions on Visualization and Computer Graphics"},{"key":"6548_CR22","unstructured":"Jocher, G., Chaurasia, A., & Qiu, J. Ultralytics YOLO. https:\/\/github.com\/ultralytics\/ultralytics"},{"key":"6548_CR23","doi-asserted-by":"crossref","unstructured":"Kalli, S., Suresh, T., Prasanth, A., Muthumanickam, T., & Mohanram, K. (2021). An effective motion object detection using adaptive background modeling mechanism in video surveillance system. Journal of Intelligent & Fuzzy Systems (Preprint), 1\u201313.","DOI":"10.3233\/JIFS-210563"},{"key":"6548_CR24","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1016\/j.vlsi.2021.01.002","volume":"78","author":"E Khatab","year":"2021","unstructured":"Khatab, E., Onsy, A., Varley, M., & Abouelfarag, A. (2021). Vulnerable objects detection for autonomous driving: A review. Integration, 78, 36\u201348.","journal-title":"Integration"},{"key":"6548_CR25","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Wu, Y., He, K., & Girshick, R. (2020). Pointrend: Image segmentation as rendering. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 9799\u20139808).","DOI":"10.1109\/CVPR42600.2020.00982"},{"key":"6548_CR26","unstructured":"Lee, M., & Kolter, Z.(2019). On physical adversarial patches for object detection. arXiv preprint arXiv:1906.11897"},{"key":"6548_CR27","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., & Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740\u2013755). Springer.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"6548_CR28","doi-asserted-by":"crossref","unstructured":"Liu, J., Levine, A., Lau, C. P., Chellappa, R., & Feizi, S. (2022). Segment and complete: Defending object detectors against adversarial patch attacks with robust patch detection. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 14973\u201314982).","DOI":"10.1109\/CVPR52688.2022.01455"},{"key":"6548_CR29","unstructured":"Liu, X., Yang, H., Liu, Z., Song, L., Li, H., & Chen, Y. (2018). Dpatch: An adversarial patch attack on object detectors. arXiv preprint arXiv:1806.02299"},{"issue":"2","key":"6548_CR30","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","volume":"60","author":"DG Lowe","year":"2004","unstructured":"Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91\u2013110.","journal-title":"International Journal of Computer Vision"},{"key":"6548_CR32","unstructured":"Lu, J., Sibai, H., Fabry, E., & Forsyth, D.(2017a). No need to worry about adversarial examples in object detection in autonomous vehicles. arXiv preprint arXiv:1707.03501"},{"key":"6548_CR33","unstructured":"Lu, J., Sibai, H., Fabry, E., & Forsyth, D. (2017b). Standard detectors aren\u2019t (currently) fooled by physical adversarial stop signs. arXiv preprint arXiv:1710.03337"},{"key":"6548_CR31","doi-asserted-by":"crossref","unstructured":"Lu, X., Li, B., Yue, Y., Li, Q., & Yan, J. (2019). Grid r-cnn. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2019.00754"},{"key":"6548_CR34","doi-asserted-by":"crossref","unstructured":"Melek, C. G., Sonmez, E. B., & Albayrak, S. (2017). A survey of product recognition in shelf images. In 2017 International conference on computer science and engineering (UBMK) (pp. 145\u2013 150). IEEE.","DOI":"10.1109\/UBMK.2017.8093584"},{"key":"6548_CR35","unstructured":"Molnar, C. (2020). Interpretable machine learning. Lulu. com."},{"issue":"7","key":"6548_CR36","first-page":"262","volume":"21","author":"J-S Oh","year":"2020","unstructured":"Oh, J.-S., & Chun, I.-G. (2020). Implementation of smart shopping cart using object detection method based on deep learning. Journal of the Korea Academia-Industrial cooperation Society, 21(7), 262\u2013269.","journal-title":"Journal of the Korea Academia-Industrial cooperation Society"},{"key":"6548_CR37","unstructured":"Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767"},{"key":"6548_CR38","unstructured":"Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28."},{"key":"6548_CR39","doi-asserted-by":"publisher","first-page":"42200","DOI":"10.1109\/ACCESS.2020.2976199","volume":"8","author":"R Roscher","year":"2020","unstructured":"Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020). Explainable machine learning for scientific insights and discoveries. IEEE Access, 8, 42200\u201342216.","journal-title":"IEEE Access"},{"key":"6548_CR40","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.imavis.2019.03.005","volume":"86","author":"B Santra","year":"2019","unstructured":"Santra, B., & Mukherjee, D. P. (2019). A comprehensive survey on computer vision based approaches for automatic identification of products in retail store. Image and Vision Computing, 86, 45\u201363.","journal-title":"Image and Vision Computing"},{"key":"6548_CR41","unstructured":"Shapira, A., Zolfi, A., Demetrio, L., Biggio, B., & Shabtai, A. (2022). Denial-of-service attack on object detection model using universal adversarial perturbation. arXiv preprint arXiv:2205.13618"},{"key":"6548_CR42","unstructured":"Song, D., Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Tramer, F., Prakash, A., & Kohno, T. (2018). Physical adversarial examples for object detectors. In 12th USENIX workshop on offensive technologies (WOOT 18)."},{"key":"6548_CR43","doi-asserted-by":"crossref","unstructured":"Thys, S., Van\u00a0Ranst, W., & Goedem\u00e9, T. (2019). Fooling automated surveillance cameras: Adversarial patches to attack person detection. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition workshops (pp. 0\u20130).","DOI":"10.1109\/CVPRW.2019.00012"},{"key":"6548_CR44","unstructured":"Vu, T., Jang, H., Pham, T. X., & Yoo, C. (2019). Cascade rpn: Delving into high-quality region proposal network with adaptive convolution. Advances in Neural Information Processing Systems, 32."},{"key":"6548_CR45","doi-asserted-by":"crossref","unstructured":"Xiang, C., & Mittal, P. (2021). Detectorguard: Provably securing object detectors against localized patch hiding attacks. In Proceedings of the 2021 ACM SIGSAC conference on computer and communications security (pp. 3177\u20133196).","DOI":"10.1145\/3460120.3484757"},{"key":"6548_CR46","doi-asserted-by":"crossref","unstructured":"Xu, K., Xiao, Y., Zheng, Z., Cai, K., & Nevatia, R. (2022). Patchzero: defending against adversarial patch attacks by detecting and zeroing the patch.","DOI":"10.1109\/WACV56688.2023.00461"},{"key":"6548_CR47","doi-asserted-by":"publisher","first-page":"6639","DOI":"10.1609\/aaai.v34i04.6140","volume":"34","author":"P Yang","year":"2020","unstructured":"Yang, P., Chen, J., Hsieh, C.-J., Wang, J.-L., & Jordan, M. (2020). Ml-loo: Detecting adversarial examples with feature attribution. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 6639\u20136647.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"6548_CR48","unstructured":"Zhang, H., Chen, H., Song, Z., Boning, D., Dhillon, I. S., & Hsieh, C.-J. (2019). The limitations of adversarial training and the blind-spot attack. arXiv preprint arXiv:1901.04684"},{"key":"6548_CR49","unstructured":"Zhu, Z., Su, H., Liu, C., Xiang, W., & Zheng, S. (2021). You cannot easily catch me: A low-detectable adversarial patch for object detectors. arXiv preprint arXiv:2109.15177"},{"key":"6548_CR50","doi-asserted-by":"crossref","unstructured":"Zolfi, A., Kravchik, M., Elovici, Y., & Shabtai, A. (2021). The translucent patch: A physical and universal attack on object detectors. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 15232\u201315241).","DOI":"10.1109\/CVPR46437.2021.01498"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06548-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06548-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06548-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T17:26:09Z","timestamp":1723051569000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06548-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,19]]},"references-count":50,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["6548"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06548-5","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,19]]},"assertion":[{"value":"26 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 March 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 April 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 June 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"All the images taken and used for this research that are not taken from publicly available datasets were filmed by the authors. In the case of images that include a human (e.g., hands and legs), the human is one of the authors and all the authors gave their consent.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}