{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T03:58:24Z","timestamp":1777694304275,"version":"3.51.4"},"reference-count":44,"publisher":"SAGE Publications","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ICA"],"published-print":{"date-parts":[[2024,1,30]]},"abstract":"<jats:p>The 3D point cloud deep neural network (3D DNN) has achieved remarkable success, but its black-box nature hinders its application in many safety-critical domains. The saliency map technique is a key method to look inside the black-box and determine where a 3D DNN focuses when recognizing a point cloud. Existing point-wise point cloud saliency methods are proposed to illustrate the point-wise saliency for a given 3D DNN. However, the above critical points are alternative and unreliable. The findings are grounded on our experimental results which show that a point becomes critical because it is responsible for representing one specific local structure. However, one local structure does not have to be represented by some specific points, conversely. As a result, discussing the saliency of the local structure (named patch-wise saliency) represented by critical points is more meaningful than discussing the saliency of some specific points. Based on the above motivations, this paper designs a black-box algorithm to generate patch-wise saliency map for point clouds. Our basic idea is to design the Mask Building-Dropping process, which adaptively matches the size of important\/unimportant patches by clustering points with close saliency. Experimental results on several typical 3D DNNs show that our patch-wise saliency algorithm can provide better visual guidance, and can detect where a 3D DNN is focusing more efficiently than a point-wise saliency map. Finally, we apply our patch-wise saliency map to adversarial attacks and backdoor defenses. The results show that the improvement is significant.<\/jats:p>","DOI":"10.3233\/ica-230725","type":"journal-article","created":{"date-parts":[[2023,12,29]],"date-time":"2023-12-29T11:53:44Z","timestamp":1703850824000},"page":"197-212","source":"Crossref","is-referenced-by-count":11,"title":["Look inside 3D point cloud deep neural network by patch-wise saliency map"],"prefix":"10.1177","volume":"31","author":[{"given":"Linkun","family":"Fan","sequence":"first","affiliation":[]},{"given":"Fazhi","family":"He","sequence":"additional","affiliation":[]},{"given":"Yupeng","family":"Song","sequence":"additional","affiliation":[]},{"given":"Huangxinxin","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Bing","family":"Li","sequence":"additional","affiliation":[]}],"member":"179","reference":[{"issue":"3","key":"10.3233\/ICA-230725_ref1","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1007\/s00371-021-02391-0","article-title":"A novel partial point cloud registration method based on graph attention network","volume":"39","author":"Song","year":"2023","journal-title":"The Visual Computer"},{"issue":"2","key":"10.3233\/ICA-230725_ref2","doi-asserted-by":"crossref","first-page":"175","DOI":"10.3233\/ICA-210674","article-title":"Image binarization method for markers tracking in extreme light conditions","volume":"29","author":"\u0106urkovi\u0107","year":"2022","journal-title":"Integrated Computer-Aided Engineering"},{"key":"10.3233\/ICA-230725_ref3","doi-asserted-by":"crossref","first-page":"105815","DOI":"10.1016\/j.engappai.2022.105815","article-title":"LiDAR-camera fusion: Dual transformer enhancement for 3D object detection","volume":"120","author":"Chen","year":"2023","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"10.3233\/ICA-230725_ref4","doi-asserted-by":"crossref","first-page":"105554","DOI":"10.1016\/j.engappai.2022.105554","article-title":"Sampling-attention deep learning network with transfer learning for large-scale urban point cloud semantic segmentation","volume":"117","author":"Zhou","year":"2023","journal-title":"Engineering Applications of Artificial Intelligence"},{"issue":"4","key":"10.3233\/ICA-230725_ref5","doi-asserted-by":"crossref","first-page":"327","DOI":"10.3233\/ICA-230710","article-title":"3D Reconstruction based on Hierarchical Reinforcement Learning with Transferability","volume":"30","author":"Lan","year":"2023","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"1","key":"10.3233\/ICA-230725_ref6","doi-asserted-by":"crossref","first-page":"14864","DOI":"10.1038\/s41598-022-19212-6","article-title":"3D convolutional neural network for machining feature recognition with gradient-based visual explanations from 3D CAD models","volume":"12","author":"Lee","year":"2022","journal-title":"Scientific Reports"},{"issue":"2","key":"10.3233\/ICA-230725_ref7","doi-asserted-by":"crossref","first-page":"141","DOI":"10.3233\/ICA-210672","article-title":"Perceptual metric-guided human image generation","volume":"29","author":"Wu","year":"2022","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"5","key":"10.3233\/ICA-230725_ref8","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1111\/mice.12906","article-title":"3D vision-based out-of-plane displacement quantification for steel plate structures using structure-from-motion, deep learning, and point-cloud processing","volume":"38","author":"Pan","year":"2023","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"issue":"1","key":"10.3233\/ICA-230725_ref9","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1111\/mice.12699","article-title":"Automated extraction of structural beam lines and connections from point clouds of steel buildings","volume":"37","author":"Smith","year":"2022","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"issue":"01","key":"10.3233\/ICA-230725_ref10","doi-asserted-by":"crossref","first-page":"2250059","DOI":"10.1142\/S0129065722500599","article-title":"Mixture 2D convolutions for 3D medical image segmentation","volume":"33","author":"Wang","year":"2023","journal-title":"International Journal of Neural Systems"},{"issue":"12","key":"10.3233\/ICA-230725_ref11","doi-asserted-by":"crossref","first-page":"2250048","DOI":"10.1142\/S0129065722500484","article-title":"Personalized watch-based fall detection using a collaborative edge-cloud framework","volume":"32","author":"Ngu","year":"2022","journal-title":"International Journal of Neural Systems"},{"issue":"2","key":"10.3233\/ICA-230725_ref12","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1111\/mice.12707","article-title":"Registration-free point cloud generation technique using rotating mirrors","volume":"37","author":"Li","year":"2022","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"key":"10.3233\/ICA-230725_ref16","doi-asserted-by":"crossref","first-page":"102219","DOI":"10.1016\/j.cagd.2023.102219","article-title":"TPNet: A novel mesh analysis method via topology preservation and perception enhancement","volume":"104","author":"Li","year":"2023","journal-title":"Computer Aided Geometric Design"},{"issue":"01","key":"10.3233\/ICA-230725_ref25","doi-asserted-by":"crossref","first-page":"2150055","DOI":"10.1142\/S0129065721500556","article-title":"Deep multimodal neural network based on data-feature fusion for patient-specific quality assurance","volume":"32","author":"Hu","year":"2022","journal-title":"International Journal of Neural Systems"},{"issue":"02","key":"10.3233\/ICA-230725_ref26","doi-asserted-by":"crossref","first-page":"2150058","DOI":"10.1142\/S0129065721500581","article-title":"Epileptic seizure prediction using deep transformer model","volume":"32","author":"Bhattacharya","year":"2022","journal-title":"International Journal of Neural Systems"},{"issue":"05","key":"10.3233\/ICA-230725_ref27","doi-asserted-by":"crossref","first-page":"2250021","DOI":"10.1142\/S0129065722500216","article-title":"Deep learning methods for multi-channel EEG-based emotion recognition","volume":"32","author":"Olamat","year":"2022","journal-title":"International Journal of Neural Systems"},{"issue":"07","key":"10.3233\/ICA-230725_ref28","doi-asserted-by":"crossref","first-page":"2250032","DOI":"10.1142\/S0129065722500320","article-title":"Epileptic seizure prediction using deep neural networks via transfer learning and multi-feature fusion","volume":"32","author":"Yu","year":"2022","journal-title":"International Journal of Neural Systems"},{"issue":"2","key":"10.3233\/ICA-230725_ref29","doi-asserted-by":"crossref","first-page":"237","DOI":"10.14359\/51689560","article-title":"Supervised deep restricted Boltzmann machine for estimation of concrete","volume":"114","author":"Rafiei","year":"2017","journal-title":"ACI Materials Journal"},{"issue":"6","key":"10.3233\/ICA-230725_ref30","doi-asserted-by":"crossref","first-page":"e12647","DOI":"10.1111\/exsy.12647","article-title":"Deep learning techniques for recommender systems based on collaborative filtering","volume":"37","author":"Martins","year":"2020","journal-title":"Expert Systems"},{"issue":"8","key":"10.3233\/ICA-230725_ref31","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1515\/revneuro-2020-0043","article-title":"Machine learning (ML) for the diagnosis of autism spectrum disorder (ASD) using brain imaging","volume":"31","author":"Nogay","year":"2020","journal-title":"Reviews in the Neurosciences"},{"issue":"6","key":"10.3233\/ICA-230725_ref32","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1159\/000512985","article-title":"Detection of epileptic seizure using pretrained deep convolutional neural network and transfer learning","volume":"83","author":"Nogay","year":"2021","journal-title":"European Neurology"},{"key":"10.3233\/ICA-230725_ref33","doi-asserted-by":"crossref","first-page":"104234","DOI":"10.1016\/j.bspc.2022.104234","article-title":"Diagnostic of autism spectrum disorder based on structural brain MRI images using, grid search optimization, and convolutional neural networks","volume":"79","author":"Nogay","year":"2023","journal-title":"Biomedical Signal Processing and Control"},{"key":"10.3233\/ICA-230725_ref34","first-page":"1","article-title":"Optimized instance segmentation by super-resolution and maximal clique generation","author":"Garc\u00eda-Aguilar","year":"2023","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"03","key":"10.3233\/ICA-230725_ref36","doi-asserted-by":"crossref","first-page":"2350010","DOI":"10.1142\/S0129065723500107","article-title":"An evolutionary attention-based network for medical image classification","volume":"33","author":"Zhu","year":"2023","journal-title":"International Journal of Neural Systems"},{"key":"10.3233\/ICA-230725_ref38","first-page":"1","article-title":"Automated detection of vehicles with anomalous trajectories in traffic surveillance videos","author":"Fern\u00e1ndez-Rodr\u00edguez","year":"2023","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"1","key":"10.3233\/ICA-230725_ref45","first-page":"17","article-title":"A deep-learning-based approach for saliency determination on point clouds","volume":"27","author":"Souai","year":"2022","journal-title":"Engineering Proceedings"},{"issue":"1","key":"10.3233\/ICA-230725_ref47","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1111\/mice.12692","article-title":"Dynamics-based cross-domain structural damage detection through deep transfer learning","volume":"37","author":"Lin","year":"2022","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"issue":"1","key":"10.3233\/ICA-230725_ref48","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1111\/mice.12694","article-title":"A deep reinforcement learning approach to mountain railway alignment optimization","volume":"37","author":"Gao","year":"2022","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"issue":"2","key":"10.3233\/ICA-230725_ref49","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1111\/mice.12701","article-title":"Deep semantic segmentation for visual understanding on construction sites","volume":"37","author":"Wang","year":"2022","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"issue":"2","key":"10.3233\/ICA-230725_ref50","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1111\/mice.12710","article-title":"Hybrid deep learning architecture for rail surface segmentation and surface defect detection","volume":"37","author":"Wu","year":"2022","journal-title":"Computer-Aided Civil and Infrastructure Engineering"},{"issue":"03","key":"10.3233\/ICA-230725_ref51","doi-asserted-by":"crossref","first-page":"2350013","DOI":"10.1142\/S0129065723500132","article-title":"Large-scale image retrieval with deep attentive global features","volume":"33","author":"Zhu","year":"2023","journal-title":"International Journal of Neural Systems"},{"issue":"2","key":"10.3233\/ICA-230725_ref53","doi-asserted-by":"crossref","first-page":"103225","DOI":"10.1016\/j.ipm.2022.103225","article-title":"Evaluating the faithfulness of saliency maps in explaining deep learning models using realistic perturbations","volume":"60","author":"Amorim","year":"2023","journal-title":"Information Processing & Management"},{"issue":"17","key":"10.3233\/ICA-230725_ref55","doi-asserted-by":"crossref","first-page":"6516","DOI":"10.3390\/s22176516","article-title":"Rethinking gradient weight influence over saliency map estimation","volume":"22","author":"Fahim","year":"2022","journal-title":"Sensors"},{"issue":"05","key":"10.3233\/ICA-230725_ref57","doi-asserted-by":"crossref","first-page":"2350026","DOI":"10.1142\/S0129065723500260","article-title":"A method based on evolutionary algorithms and channel attention mechanism to enhance cycle generative adversarial network performance for image translation","volume":"33","author":"Xue","year":"2023","journal-title":"International Journal of Neural Systems"},{"key":"10.3233\/ICA-230725_ref61","doi-asserted-by":"crossref","first-page":"106987","DOI":"10.1016\/j.patcog.2019.106987","article-title":"Video saliency detection by gestalt theory","volume":"96","author":"Fang","year":"2019","journal-title":"Pattern Recognition"},{"issue":"5","key":"10.3233\/ICA-230725_ref65","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3326362","article-title":"Dynamic graph cnn for learning on point clouds","volume":"38","author":"Wang","year":"2019","journal-title":"Acm Transactions On Graphics (tog)"},{"key":"10.3233\/ICA-230725_ref70","doi-asserted-by":"crossref","first-page":"14410","DOI":"10.1109\/ACCESS.2018.2807385","article-title":"Threat of adversarial attacks on deep learning in computer vision: A survey","volume":"6","author":"Akhtar","year":"2018","journal-title":"Ieee Access"},{"key":"10.3233\/ICA-230725_ref71","doi-asserted-by":"crossref","first-page":"109466","DOI":"10.1016\/j.patcog.2023.109466","article-title":"Adversarial pan-sharpening attacks for object detection in remote sensing","volume":"139","author":"Wei","year":"2023","journal-title":"Pattern Recognition"},{"key":"10.3233\/ICA-230725_ref73","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.soildyn.2017.05.013","article-title":"NEEWS: A novel earthquake early warning model using neural dynamic classification and neural dynamic optimization","volume":"100","author":"Rafiei","year":"2017","journal-title":"Soil Dynamics and Earthquake Engineering"},{"issue":"12","key":"10.3233\/ICA-230725_ref74","doi-asserted-by":"crossref","first-page":"04018106","DOI":"10.1061\/(ASCE)CO.1943-7862.0001570","article-title":"Novel machine learning model for construction cost estimation taking into account economic variables and indices","volume":"144","author":"Rafiei","year":"2018","journal-title":"Journal of Construction Engineering and Management"},{"key":"10.3233\/ICA-230725_ref76","first-page":"22103","article-title":"Finding naturally occurring physical backdoors in image datasets","volume":"35","author":"Wenger","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"6","key":"10.3233\/ICA-230725_ref78","doi-asserted-by":"crossref","first-page":"e12494","DOI":"10.1111\/exsy.12494","article-title":"A novel end-to-end deep learning scheme for classifying multi-class motor imagery electroencephalography signals","volume":"36","author":"Hassanpour","year":"2019","journal-title":"Expert Systems"},{"issue":"7","key":"10.3233\/ICA-230725_ref87","doi-asserted-by":"crossref","first-page":"1862","DOI":"10.1109\/TMM.2020.2987694","article-title":"Exploiting vulnerabilities of deep neural networks for privacy protection","volume":"22","author":"Sanchez-Matilla","year":"2020","journal-title":"IEEE Transactions on Multimedia"},{"key":"10.3233\/ICA-230725_ref89","first-page":"10546","article-title":"Backdoorbench: A comprehensive benchmark of backdoor learning","volume":"35","author":"Wu","year":"2022","journal-title":"Advances in Neural Information Processing Systems"}],"container-title":["Integrated Computer-Aided Engineering"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/ICA-230725","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:14:51Z","timestamp":1777454091000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/ICA-230725"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,30]]},"references-count":44,"journal-issue":{"issue":"2"},"URL":"https:\/\/doi.org\/10.3233\/ica-230725","relation":{},"ISSN":["1069-2509","1875-8835"],"issn-type":[{"value":"1069-2509","type":"print"},{"value":"1875-8835","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,30]]}}}