{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T00:06:12Z","timestamp":1785197172543,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,12,27]],"date-time":"2018-12-27T00:00:00Z","timestamp":1545868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-15-1-0442"],"award-info":[{"award-number":["FA9550-15-1-0442"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,12,27]]},"DOI":"10.1145\/3278721.3278776","type":"proceedings-article","created":{"date-parts":[[2019,1,10]],"date-time":"2019-01-10T13:40:41Z","timestamp":1547127641000},"page":"144-150","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":97,"title":["Transparency and Explanation in Deep Reinforcement Learning Neural Networks"],"prefix":"10.1145","author":[{"given":"Rahul","family":"Iyer","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuezhang","family":"Li","sequence":"additional","affiliation":[{"name":"Google Inc., Mountain View, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huao","family":"Li","sequence":"additional","affiliation":[{"name":"University of Pittsburgh, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Lewis","sequence":"additional","affiliation":[{"name":"University of Pittsburgh, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ramitha","family":"Sundar","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katia","family":"Sycara","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,12,27]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1080\/00140130701217149"},{"key":"e_1_3_2_1_2_1","volume-title":"Template Matching Techniques in Computer Vision: Theory and Practice","author":"Brunelli Roberto","unstructured":"Roberto Brunelli . 2009. Template Matching Techniques in Computer Vision: Theory and Practice . Wiley Publishing . Roberto Brunelli. 2009. Template Matching Techniques in Computer Vision: Theory and Practice .Wiley Publishing."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07458-0_24"},{"key":"e_1_3_2_1_5_1","first-page":"3","article-title":"Visualizing higher-layer features of a deep network","volume":"1341","author":"Erhan Dumitru","year":"2009","unstructured":"Dumitru Erhan , Yoshua Bengio , Aaron Courville , and Pascal Vincent . 2009 . Visualizing higher-layer features of a deep network . University of Montreal , Vol. 1341 (2009), 3 . Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. University of Montreal , Vol. 1341 (2009), 3.","journal-title":"University of Montreal"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2005.10.010"},{"key":"e_1_3_2_1_7_1","volume-title":"Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580","author":"Hinton Geoffrey E","year":"2012","unstructured":"Geoffrey E Hinton , Nitish Srivastava , Alex Krizhevsky , Ilya Sutskever , and Ruslan R Salakhutdinov . 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 ( 2012 ). Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)."},{"key":"e_1_3_2_1_8_1","unstructured":"Itseez. 2015. Open Source Computer Vision Library. https:\/\/github.com\/itseez\/opencv .  Itseez. 2015. Open Source Computer Vision Library. https:\/\/github.com\/itseez\/opencv ."},{"key":"e_1_3_2_1_9_1","unstructured":"Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.   Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105."},{"key":"e_1_3_2_1_10_1","volume-title":"Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 8595--8598","author":"Le Quoc V","year":"2013","unstructured":"Quoc V Le . 2013 . Building high-level features using large scale unsupervised learning. In Acoustics , Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 8595--8598 . Quoc V Le. 2013. Building high-level features using large scale unsupervised learning. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 8595--8598."},{"key":"e_1_3_2_1_11_1","volume-title":"Trust in automation: Designing for appropriate reliance. Human factors","author":"Lee John D","year":"2004","unstructured":"John D Lee and Katrina A See . 2004. Trust in automation: Designing for appropriate reliance. Human factors , Vol. 46 , 1 ( 2004 ), 50--80. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors , Vol. 46, 1 (2004), 50--80."},{"key":"e_1_3_2_1_12_1","volume-title":"Object Sensitive Deep Reinforcement Learning. In 3rd Global Conference on Artificial Intelligence . EPiC Series in Computing.","author":"Li Yuezhang","year":"2017","unstructured":"Yuezhang Li , Katia Sycara , and Rahul Iyer . 2017 . Object Sensitive Deep Reinforcement Learning. In 3rd Global Conference on Artificial Intelligence . EPiC Series in Computing. Yuezhang Li, Katia Sycara, and Rahul Iyer. 2017. Object Sensitive Deep Reinforcement Learning. In 3rd Global Conference on Artificial Intelligence . EPiC Series in Computing."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/154193120605002304"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07458-0_18"},{"key":"e_1_3_2_1_15_1","volume-title":"Michael J Barnes, Daniel Barber, and Katelyn Procci.","author":"Mercado Joseph E","year":"2016","unstructured":"Joseph E Mercado , Michael A Rupp , Jessie YC Chen , Michael J Barnes, Daniel Barber, and Katelyn Procci. 2016 . Intelligent agent transparency in human--agent teaming for Multi-UxV management. Human factors , Vol. 58 , 3 (2016), 401--415. Joseph E Mercado, Michael A Rupp, Jessie YC Chen, Michael J Barnes, Daniel Barber, and Katelyn Procci. 2016. Intelligent agent transparency in human--agent teaming for Multi-UxV management. Human factors , Vol. 58, 3 (2016), 401--415."},{"key":"e_1_3_2_1_16_1","volume-title":"Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu.","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih , Adri\u00e0 Puigdom\u00e8 nech Badia , Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016 . Asynchronous Methods for Deep Reinforcement Learning. CoRR , Vol. abs\/ 1602 .01783 (2016). http:\/\/arxiv.org\/abs\/1602.01783 Volodymyr Mnih, Adri\u00e0 Puigdom\u00e8 nech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous Methods for Deep Reinforcement Learning. CoRR , Vol. abs\/1602.01783 (2016). http:\/\/arxiv.org\/abs\/1602.01783"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2006.10.001"},{"key":"e_1_3_2_1_20_1","volume-title":"Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229","author":"Sermanet Pierre","year":"2013","unstructured":"Pierre Sermanet , David Eigen , Xiang Zhang , Micha\u00ebl Mathieu , Rob Fergus , and Yann LeCun . 2013 . Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013). Pierre Sermanet, David Eigen, Xiang Zhang, Micha\u00ebl Mathieu, Rob Fergus, and Yann LeCun. 2013. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)."},{"key":"e_1_3_2_1_21_1","volume-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034","author":"Simonyan Karen","year":"2013","unstructured":"Karen Simonyan , Andrea Vedaldi , and Andrew Zisserman . 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 ( 2013 ). Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)."},{"key":"e_1_3_2_1_22_1","volume-title":"et almbox","author":"Song Hyun Oh","year":"2014","unstructured":"Hyun Oh Song , Ross B Girshick , Stefanie Jegelka , Julien Mairal , Zaid Harchaoui , Trevor Darrell , et almbox . 2014 . On learning to localize objects with minimal supervision.. In ICML . 1611--1619. Hyun Oh Song, Ross B Girshick, Stefanie Jegelka, Julien Mairal, Zaid Harchaoui, Trevor Darrell, et almbox. 2014. On learning to localize objects with minimal supervision.. In ICML . 1611--1619."},{"key":"e_1_3_2_1_23_1","volume-title":"International journal of vehicle design","author":"Stanton Neville A","year":"2007","unstructured":"Neville A Stanton , Mark S Young , and Guy H Walker . 2007. The psychology of driving automation: a discussion with Professor Don Norman . International journal of vehicle design , Vol. 45 , 3 ( 2007 ), 289--306. Neville A Stanton, Mark S Young, and Guy H Walker. 2007. The psychology of driving automation: a discussion with Professor Don Norman. International journal of vehicle design , Vol. 45, 3 (2007), 289--306."},{"key":"e_1_3_2_1_24_1","unstructured":"Richard S Sutton. 1996. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Advances in neural information processing systems. 1038--1044.   Richard S Sutton. 1996. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Advances in neural information processing systems. 1038--1044."},{"key":"e_1_3_2_1_25_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton Richard S","unstructured":"Richard S Sutton and Andrew G Barto . 1998. Reinforcement learning: An introduction . Vol. 1 . MIT press Cambridge . Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An introduction . Vol. 1. MIT press Cambridge."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_1_27_1","volume-title":"Deep Reinforcement Learning with Double Q-learning. CoRR","author":"van Hasselt Hado","year":"2015","unstructured":"Hado van Hasselt , Arthur Guez , and David Silver . 2015. Deep Reinforcement Learning with Double Q-learning. CoRR , Vol. abs\/ 1509 .06461 ( 2015 ). http:\/\/arxiv.org\/abs\/1509.06461 Hado van Hasselt, Arthur Guez, and David Silver. 2015. Deep Reinforcement Learning with Double Q-learning. CoRR , Vol. abs\/1509.06461 (2015). http:\/\/arxiv.org\/abs\/1509.06461"},{"key":"e_1_3_2_1_28_1","volume-title":"Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581","author":"Wang Ziyu","year":"2015","unstructured":"Ziyu Wang , Nando de Freitas , and Marc Lanctot . 2015. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581 ( 2015 ). Ziyu Wang, Nando de Freitas, and Marc Lanctot. 2015. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581 (2015)."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10590-1_53"}],"event":{"name":"AIES '18: AAAI\/ACM Conference on AI, Ethics, and Society","location":"New Orleans LA USA","acronym":"AIES '18","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence"]},"container-title":["Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3278721.3278776","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3278721.3278776","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3278721.3278776","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:58:00Z","timestamp":1750208280000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3278721.3278776"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,27]]},"references-count":29,"alternative-id":["10.1145\/3278721.3278776","10.1145\/3278721"],"URL":"https:\/\/doi.org\/10.1145\/3278721.3278776","relation":{},"subject":[],"published":{"date-parts":[[2018,12,27]]},"assertion":[{"value":"2018-12-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}