{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:25:09Z","timestamp":1750220709001,"version":"3.41.0"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2020,5,3]],"date-time":"2020-05-03T00:00:00Z","timestamp":1588464000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National High-level Personnel for Defense Technology Program","award":["2017-JCJQ-ZQ-013"],"award-info":[{"award-number":["2017-JCJQ-ZQ-013"]}]},{"name":"NSF China","award":["61902405"],"award-info":[{"award-number":["61902405"]}]},{"name":"D2DCRC","award":["DC25002,DC25003"],"award-info":[{"award-number":["DC25002,DC25003"]}]},{"name":"ARC DPs","award":["170103710,180103411"],"award-info":[{"award-number":["170103710,180103411"]}]},{"name":"HUNAN Province Science Foundation","award":["2017RS3045"],"award-info":[{"award-number":["2017RS3045"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Web"],"published-print":{"date-parts":[[2020,8,31]]},"abstract":"<jats:p>Sharing a pre-trained machine learning model, particularly a deep neural network via prediction APIs, is becoming a common practice on machine learning as a service (MLaaS) platforms nowadays. Although deep neural networks (DNN) have shown remarkable successes in many tasks, they are also criticized for the lack of interpretability and transparency. Interpreting a shared DNN model faces two additional challenges compared with interpreting a general model. (1) Limited training data can be disclosed to users. (2) The internal structure of the models may not be available. These two challenges impede the application of most existing interpretability approaches, such as saliency maps or influence functions, for DNN models. Case-based reasoning methods have been used for interpreting decisions; however, how to select and organize the data points under the constraints of shared DNN models is not discussed. Moreover, simply providing cases as explanations may not be sufficient for supporting instance level interpretability. Meanwhile, existing interpretation methods for DNN models generally lack the means to evaluate the reliability of the interpretation. In this article, we propose a framework named Shared Model INTerpreter (SMINT) to address the above limitations. We propose a new data structure called a boundary graph to organize training points to mimic the predictions of DNN models. We integrate local features, such as saliency maps and interpretable input masks, into the data structure to help users to infer the model decision boundaries. We show that the boundary graph is able to address the reliability issues in many local interpretation methods. We further design an algorithm named hidden-layer aware p-test to measure the reliability of the interpretations. Our experiments show that SMINT is able to achieve above 99% fidelity to corresponding DNN models on both MNIST and ImageNet by sharing only a tiny fraction of training data to make these models interpretable. The human pilot study demonstrates that SMINT provides better interpretability compared with existing methods. Moreover, we demonstrate that SMINT is able to assist model tuning for better performance on different user data.<\/jats:p>","DOI":"10.1145\/3381833","type":"journal-article","created":{"date-parts":[[2020,5,4]],"date-time":"2020-05-04T23:50:36Z","timestamp":1588636236000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["SMINT"],"prefix":"10.1145","volume":"14","author":[{"given":"Huijun","family":"Wu","sequence":"first","affiliation":[{"name":"National University of Defense Technology, China and UNSW, China, UNSW, Sydney, Australia"}]},{"given":"Chen","family":"Wang","sequence":"additional","affiliation":[{"name":"Data61, CSIRO, Garden Street, Eveleigh, NSW, Sydney, Australia"}]},{"given":"Richard","family":"Nock","sequence":"additional","affiliation":[{"name":"Data61, CSIRO, Garden Street, Eveleigh, NSW, Sydney, Australia"}]},{"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"University of New South Wales, Kensington, NSW, Sydney, Australia"}]},{"given":"Jie","family":"Yin","sequence":"additional","affiliation":[{"name":"University of Sydney, NSW, Sydney, Australia"}]},{"given":"Kai","family":"Lu","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Kaifu, Changsha, China"}]},{"given":"Liming","family":"Zhu","sequence":"additional","affiliation":[{"name":"Data61, CSIRO and University of New South Wales, Eveleigh, NSW, Sydney, Australia"}]}],"member":"320","published-online":{"date-parts":[[2020,5,3]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.3233\/AIC-1994-7104"},{"key":"e_1_2_1_2_1","unstructured":"Julius Adebayo Justin Gilmer Michael Muelly Ian Goodfellow Moritz Hardt and Been Kim. 2018. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems. 9505--9515.  Julius Adebayo Justin Gilmer Michael Muelly Ian Goodfellow Moritz Hardt and Been Kim. 2018. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems. 9505--9515."},{"volume-title":"Jaakkola","year":"2018","author":"Alvarez-Melis David","key":"e_1_2_1_3_1"},{"volume-title":"Machine Learning on AWS. Retrieved","year":"2017","key":"e_1_2_1_4_1"},{"key":"e_1_2_1_5_1","unstructured":"Alexandr Andoni Piotr Indyk Thijs Laarhoven Ilya Razenshteyn and Ludwig Schmidt. 2015. Practical and optimal LSH for angular distance. In Advances in Neural Information Processing Systems. 1225--1233.  Alexandr Andoni Piotr Indyk Thijs Laarhoven Ilya Razenshteyn and Ludwig Schmidt. 2015. Practical and optimal LSH for angular distance. In Advances in Neural Information Processing Systems. 1225--1233."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2746539.2746553"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/116873.116880"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1214\/11-AOAS495"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01228508"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775113"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3128572.3140444"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the AMIA Annual Symposium Proceedings","volume":"2016","author":"Che Zhengping","year":"2016"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/PL00009897"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1177\/001872089606380203"},{"volume-title":"Shavlik","year":"1996","author":"Craven Mark","key":"e_1_2_1_16_1"},{"volume-title":"Proceedings of the 20th Annual Symposium on Computational Geometry. ACM, 253--262","author":"Datar Mayur","key":"e_1_2_1_17_1"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.371"},{"key":"e_1_2_1_19_1","unstructured":"Nicholas Frosst and Geoffrey Hinton. 2017. Distilling a neural network into a soft decision tree. https:\/\/arxiv.org\/pdf\/1711.09784.pdf.  Nicholas Frosst and Geoffrey Hinton. 2017. Distilling a neural network into a soft decision tree. https:\/\/arxiv.org\/pdf\/1711.09784.pdf."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013681"},{"key":"e_1_2_1_21_1","unstructured":"Ian J Goodfellow Jonathon Shlens and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. https:\/\/arxiv.org\/abs\/1412.6572.pdf.  Ian J Goodfellow Jonathon Shlens and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. https:\/\/arxiv.org\/abs\/1412.6572.pdf."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2205597"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-017-0472-7"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299017"},{"volume-title":"Koyejo","year":"2016","author":"Kim Been","key":"e_1_2_1_27_1"},{"volume-title":"Proceedings of the International Conference on Machine Learning. 2673--2682","year":"2018","author":"Kim Been","key":"e_1_2_1_28_1"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"volume-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning","author":"Kindermans Pieter-Jan","key":"e_1_2_1_30_1"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning","volume":"70","author":"Koh Pang Wei","year":"2017"},{"volume-title":"Proceedings of the 11th Annual Conference Cognitive Science Society Pod. 155--162","year":"2014","author":"Kolodner J.","key":"e_1_2_1_32_1"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00155578"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459466"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00977785"},{"volume-title":"Thirty-Second AAAI Conference on Artificial Intelligence.","year":"2018","author":"Li Oscar","key":"e_1_2_1_37_1"},{"volume-title":"Proceedings of the International Conference on Learning Representations (ICLR\u201918)","year":"2018","author":"Ma Xingjun","key":"e_1_2_1_38_1"},{"volume-title":"Proceedings of the AAAI Conference on Artifical Intelligence (AAAI\u201915)","author":"Mathy Charles","key":"e_1_2_1_39_1"},{"volume-title":"Microsoft Azure Machine Learning Studio. Retrieved","year":"2017","key":"e_1_2_1_40_1"},{"issue":"2019","key":"e_1_2_1_41_1","first-page":"1","article-title":"Explanation in artificial intelligence: Insights from the social sciences","volume":"267","author":"Miller Tim","year":"2018","journal-title":"Artificial Intelligence"},{"volume-title":"Herbert Alexander Simon, et al","year":"1972","author":"Newell Allen","key":"e_1_2_1_42_1"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218001403002952"},{"volume-title":"International Conference on Machine Learning. 4901--4911","year":"2019","author":"Odena Augustus","key":"e_1_2_1_44_1"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.3115\/1219840.1219855"},{"key":"e_1_2_1_46_1","unstructured":"Nicolas Papernot Fartash Faghri Nicholas Carlini Ian Goodfellow Reuben Feinman Alexey Kurakin Cihang Xie Yash Sharma Tom Brown Aurko Roy Alexander Matyasko Vahid Behzadan Karen Hambardzumyan Zhishuai Zhang Yi-Lin Juang Zhi Li Ryan Sheatsley Abhibhav Garg Jonathan Uesato Willi Gierke Yinpeng Dong David Berthelot Paul Hendricks Jonas Rauber and Rujun Long. 2018. Technical report on the CleverHans v2.1.0 adversarial examples library. https:\/\/arxiv.org\/pdf\/1610.00768.pdf.  Nicolas Papernot Fartash Faghri Nicholas Carlini Ian Goodfellow Reuben Feinman Alexey Kurakin Cihang Xie Yash Sharma Tom Brown Aurko Roy Alexander Matyasko Vahid Behzadan Karen Hambardzumyan Zhishuai Zhang Yi-Lin Juang Zhi Li Ryan Sheatsley Abhibhav Garg Jonathan Uesato Willi Gierke Yinpeng Dong David Berthelot Paul Hendricks Jonas Rauber and Rujun Long. 2018. Technical report on the CleverHans v2.1.0 adversarial examples library. https:\/\/arxiv.org\/pdf\/1610.00768.pdf."},{"key":"e_1_2_1_47_1","unstructured":"Nicolas Papernot and Patrick McDaniel. 2018. Deep k-nearest neighbors: Towards confident interpretable and robust deep learning. https:\/\/arxiv.org\/abs\/1803.04765.pdf.  Nicolas Papernot and Patrick McDaniel. 2018. Deep k-nearest neighbors: Towards confident interpretable and robust deep learning. https:\/\/arxiv.org\/abs\/1803.04765.pdf."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2016.36"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"volume-title":"Cloud Machine Learning Engine. Retrieved","year":"2018","author":"Platform Google Cloud","key":"e_1_2_1_50_1"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/1735958.1735960"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.809084"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"volume-title":"Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.","year":"2016","author":"Silver David","key":"e_1_2_1_56_1"},{"volume-title":"Proceedings of the International Conference on Learning Representations (ICLR\u201915)","author":"Springenberg J.","key":"e_1_2_1_57_1"},{"key":"e_1_2_1_58_1","unstructured":"Pierre Stock and Moustapha Cisse. 2017. Convnets and imagenet beyond accuracy: Explanations bias detection adversarial examples and model criticism. https:\/\/arxiv.org\/abs\/1711.11443.pdf.  Pierre Stock and Moustapha Cisse. 2017. Convnets and imagenet beyond accuracy: Explanations bias detection adversarial examples and model criticism. https:\/\/arxiv.org\/abs\/1711.11443.pdf."},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning","volume":"70","author":"Sundararajan Mukund","year":"2017"},{"volume":"4","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI\u201917)","author":"Szegedy Christian","key":"e_1_2_1_60_1"},{"volume-title":"Deep MNIST for Experts. Retrieved","year":"2017","key":"e_1_2_1_61_1"},{"volume-title":"Goodman","year":"2004","author":"Toth Csaba D.","key":"e_1_2_1_62_1"},{"volume-title":"Proceedings of the USENIX Security Conference.","year":"2016","author":"Tram\u00e8r Florian","key":"e_1_2_1_63_1"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3185995"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11501"},{"volume-title":"Proceedings of the Network and Distributed System Security Symposium (NDSS\u201918)","year":"2017","author":"Xu Weilin","key":"e_1_2_1_66_1"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01231-1_31"},{"volume-title":"Zeiler and Rob Fergus","year":"2014","author":"Matthew","key":"e_1_2_1_68_1"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00920"}],"container-title":["ACM Transactions on the Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3381833","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3381833","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:33:07Z","timestamp":1750199587000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3381833"}},"subtitle":["Toward Interpretable and Robust Model Sharing for Deep Neural Networks"],"short-title":[],"issued":{"date-parts":[[2020,5,3]]},"references-count":69,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,8,31]]}},"alternative-id":["10.1145\/3381833"],"URL":"https:\/\/doi.org\/10.1145\/3381833","relation":{},"ISSN":["1559-1131","1559-114X"],"issn-type":[{"type":"print","value":"1559-1131"},{"type":"electronic","value":"1559-114X"}],"subject":[],"published":{"date-parts":[[2020,5,3]]},"assertion":[{"value":"2019-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-05-03","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}