{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T16:16:37Z","timestamp":1780503397951,"version":"3.54.1"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Asian Office of Aerospace Research and Development","award":["FA2386-16-1-4071"],"award-info":[{"award-number":["FA2386-16-1-4071"]}]},{"name":"U.S. Office of Naval Research Science of Autonomy","award":["N00014-17-1-2434"],"award-info":[{"award-number":["N00014-17-1-2434"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SN COMPUT. SCI."],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A robot\u2019s ability to provide explanatory descriptions of its decisions and beliefs promotes effective collaboration with humans. Providing the desired transparency in decision making is challenging in integrated robot systems that include knowledge-based reasoning methods and data-driven learning methods. As a step towards addressing this challenge, our architecture combines the complementary strengths of non-monotonic logical reasoning with incomplete commonsense domain knowledge, deep learning, and inductive learning. During reasoning and learning, the architecture enables a robot to provide on-demand explanations of its decisions, the evolution of associated beliefs, and the outcomes of hypothetical actions, in the form of relational descriptions of relevant domain objects, attributes, and actions. The architecture\u2019s capabilities are illustrated and evaluated in the context of scene understanding tasks and planning tasks performed using simulated images and images from a physical robot manipulating tabletop objects. Experimental results indicate the ability to reliably acquire and merge new information about the domain in the form of constraints, preconditions, and effects of actions, and to provide accurate explanations in the presence of noisy sensing and actuation.<\/jats:p>","DOI":"10.1007\/s42979-021-00573-0","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T14:03:26Z","timestamp":1619705006000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics"],"prefix":"10.1007","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2339-0270","authenticated-orcid":false,"given":"Tiago","family":"Mota","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9922-8969","authenticated-orcid":false,"given":"Mohan","family":"Sridharan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0773-3277","authenticated-orcid":false,"given":"Ale\u0161","family":"Leonardis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,4,29]]},"reference":[{"key":"573_CR1","unstructured":"Anjomshoae S, Najjar A, Calvaresi D, Framling K. Explainable agents and robots: results from a systematic literature review. In: International conference on autonomous agents and multiagent systems. Montreal, Canada. 2019."},{"issue":"3","key":"573_CR2","doi-asserted-by":"publisher","first-page":"662","DOI":"10.1016\/j.datak.2007.10.006","volume":"64","author":"G Antoniou","year":"2008","unstructured":"Antoniou G, Bikakis A, Dimaresis N, Genetzakis M, Georgalis G, Governatori G, Karouzaki E, Kazepis N, Kosmadakis D, Kritsotakis M, et al. Proof explanation for a nonmonotonic semantic web rules language. Data Knowl Eng. 2008;64(3):662\u201387.","journal-title":"Data Knowl. Eng."},{"key":"573_CR3","doi-asserted-by":"crossref","unstructured":"Assaf R, Schumann A. Explainable deep neural networks for multivariate time series predictions. In: International joint conference on artificial intelligence, Macao, China, pp. 6488\u20136490. 2019.","DOI":"10.24963\/ijcai.2019\/932"},{"key":"573_CR4","doi-asserted-by":"crossref","unstructured":"Bercher P, Biundo S, Geier T, Hoernle T, Nothdurft F, Richter F, Schattenberg B. Plan, repair, execute, explain - how planning helps to assemble your home theater. In: Twenty-fourth international conference on automated planning and scheduling. 2014","DOI":"10.1609\/icaps.v24i1.13664"},{"key":"573_CR5","unstructured":"Borgo R, Cashmore M, Magazzeni D. Towards providing explanations for AI planner decisions. In: IJCAI workshop on explainable artificial intelligence, pp. 11\u201317. 2018."},{"key":"573_CR6","doi-asserted-by":"crossref","unstructured":"Chai JY, Gao Q, She L, Yang S, Saba-Sadiya S, Xu G. Language to action: towards interactive task learning with physical agents. In: International joint conference on artificial intelligence. 2018.","DOI":"10.24963\/ijcai.2018\/1"},{"issue":"2\u20133","key":"573_CR7","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1007\/s13218-018-0544-x","volume":"32","author":"E Erdem","year":"2018","unstructured":"Erdem E, Patoglu V. Applications of ASP in robotics. Kunstliche Intelligenz. 2018;32(2\u20133):143\u20139.","journal-title":"Kunstliche Intelligenz"},{"issue":"2","key":"573_CR8","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1017\/S1471068418000534","volume":"19","author":"J Fandinno","year":"2019","unstructured":"Fandinno J, Schulz C. Answering the \u201cWhy\u2019\u2019 in answer set programming: a survey of explanation approaches. Theory and Practice of Logic Programming. 2019;19(2):114\u2013203.","journal-title":"Theory and Practice of Logic Programming"},{"key":"573_CR9","first-page":"1001","volume":"25","author":"G Ferrand","year":"2006","unstructured":"Ferrand G, Lessaint W, Tessier A. Explanations and proof trees. Comput Inform. 2006;25:1001\u201321.","journal-title":"Comput. Inform."},{"key":"573_CR10","unstructured":"Fox M, Long D, Magazzeni D. Explainable planning. In: IJCAI workshop on explainable AI. 2017."},{"issue":"1","key":"573_CR11","first-page":"5","volume":"71","author":"M Friedman","year":"1974","unstructured":"Friedman M. Explanation and scientific understanding. Philosophy. 1974;71(1):5\u201319.","journal-title":"Philosophy"},{"issue":"1\u20132","key":"573_CR12","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1080\/11663081.2013.798954","volume":"23","author":"M Gelfond","year":"2013","unstructured":"Gelfond M, Inclezan D. Some properties of system descriptions of $$AL_d$$. J Appl Non Class Logics Spec Issue Equilib Logic Answ Set Programm. 2013;23(1\u20132):105\u201320.","journal-title":"J Appl Non Class Logics Spec Issue Equilib Logic Answ Set Programm."},{"key":"573_CR13","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139342124","volume-title":"Knowledge representation, reasoning and the design of intelligent agents","author":"M Gelfond","year":"2014","unstructured":"Gelfond M, Kahl Y. Knowledge representation, reasoning and the design of intelligent agents. Cambridge: Cambridge University Press; 2014."},{"key":"573_CR14","doi-asserted-by":"crossref","unstructured":"Gil Y. Learning by experimentation: incremental refinement of incomplete planning domains. In: International conference on machine learning, pp. 87\u201395. 1994.","DOI":"10.1016\/B978-1-55860-335-6.50019-2"},{"issue":"5\u20136","key":"573_CR15","doi-asserted-by":"publisher","first-page":"817","DOI":"10.1017\/S1471068416000260","volume":"16","author":"N Katzouris","year":"2016","unstructured":"Katzouris N, Artikis A, Paliouras G. Online learning of event definitions. Theory Pract Logic Programm. 2016;16(5\u20136):817\u201333.","journal-title":"Theory Pract Logic Programm."},{"key":"573_CR16","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/0004-3702(87)90063-4","volume":"32","author":"J de Kleer","year":"1987","unstructured":"de Kleer J, Williams BC. Diagnosing multiple faults. Artif Intell. 1987;32:97\u2013130.","journal-title":"Artif Intell."},{"key":"573_CR17","unstructured":"Koh PW, Liang P. Understanding black-box predictions via influence functions. In: International conference on machine learning. pp. 1885\u20131894. 2017."},{"issue":"3","key":"573_CR18","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1016\/j.knosys.2010.12.001","volume":"24","author":"E Kontopoulos","year":"2011","unstructured":"Kontopoulos E, Bassiliades N, Antoniou G. Visualizing semantic web proofs of defeasible logic in the dr-device system. Knowl Based Syst. 2011;24(3):406\u201319.","journal-title":"Knowl Based Syst."},{"key":"573_CR19","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Neural information processing systems, pp. 1097\u20131105. 2012."},{"key":"573_CR20","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7688.001.0001","volume-title":"The soar cognitive architecture","author":"JE Laird","year":"2012","unstructured":"Laird JE. The soar cognitive architecture. Cambridge: The MIT Press; 2012."},{"issue":"4","key":"573_CR21","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1109\/MIS.2017.3121552","volume":"32","author":"JE Laird","year":"2017","unstructured":"Laird JE, Gluck K, Anderson J, Forbus KD, Jenkins OC, Lebiere C, Salvucci D, Scheutz M, Thomaz A, Trafton G, Wray RE, Mohan S, Kirk JR. Interactive task learning. IEEE Intell Syst. 2017;32(4):6\u201321.","journal-title":"IEEE Intell Syst."},{"key":"573_CR22","volume-title":"Innovative applications of artificial intelligence","author":"P Langley","year":"2017","unstructured":"Langley P, Meadows B, Sridharan M, Choi D. Explainable agency for intelligent autonomous systems. In: Innovative applications of artificial intelligence. Cambridge: AAAI Press; 2017."},{"key":"573_CR23","unstructured":"Law M, Russo A, Broda K. The ILASP system for inductive learning of answer set program. Technical report on arXiV. 2020. https:\/\/arxiv.org\/abs\/2005.00904."},{"issue":"11","key":"573_CR24","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86(11):2278\u2013324.","journal-title":"Proc IEEE."},{"issue":"2","key":"573_CR25","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1037\/1076-898X.6.2.104","volume":"6","author":"S Lewandowsky","year":"2000","unstructured":"Lewandowsky S, Mundy M, Tan G. The dynamics of trust: comparing humans to automation. J Exp Psychol Appl. 2000;6(2):104.","journal-title":"J Exp Psychol Appl."},{"key":"573_CR26","unstructured":"McGuinness DL, Glass A, Wolverton M, Da\u00a0Silva PP. Explaining task processing in cognitive assistants that learn. In: AAAI spring symposium: interaction challenges for intelligent assistants, pp. 80\u201387. 2007."},{"key":"573_CR27","volume-title":"The Stanford encyclopedia of philosophy","author":"P Menzies","year":"2020","unstructured":"Menzies P, Beebee H. Counterfactual theories of causation. In: Zalta EN, editor. The Stanford encyclopedia of philosophy. 2020th ed. Stanford: Stanford University; 2020.","edition":"2020"},{"issue":"11","key":"573_CR28","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1145\/219717.219748","volume":"38","author":"GA Miller","year":"1995","unstructured":"Miller GA. WordNet: a lexical database for English. Commun ACM. 1995;38(11):39\u201341.","journal-title":"Commun ACM."},{"key":"573_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","volume":"267","author":"T Miller","year":"2019","unstructured":"Miller T. Explanations in artificial intelligence: insights from the social sciences. Artif Intell. 2019;267:1\u201338.","journal-title":"Artif Intell."},{"key":"573_CR30","doi-asserted-by":"crossref","unstructured":"Mota T, Sridharan M. Incrementally grounding expressions for spatial relations between objects. In: International joint conference on artificial intelligence, pp. 1928\u20131934. 2018.","DOI":"10.24963\/ijcai.2018\/266"},{"key":"573_CR31","doi-asserted-by":"crossref","unstructured":"Mota T, Sridharan M. Commonsense reasoning and knowledge acquisition to guide deep learning on robots. In: Robotics science and systems. 2019.","DOI":"10.15607\/RSS.2019.XV.077"},{"key":"573_CR32","doi-asserted-by":"crossref","unstructured":"Mota T, Sridharan M. Answer me this: constructing disambiguation queries for explanation generation in robotics. In: Workshop of the UK planning and scheduling special interest group. 2020.","DOI":"10.1109\/ICDL49984.2021.9515605"},{"key":"573_CR33","doi-asserted-by":"crossref","unstructured":"Mota T, Sridharan M. Commonsense reasoning and deep learning for transparent decision making in robotics. In: European conference on multiagent systems. 2020.","DOI":"10.1007\/978-3-030-66412-1_14"},{"key":"573_CR34","unstructured":"Mota T, Sridharan M. Scene understanding, reasoning, and explanation generation. 2020. https:\/\/github.com\/tmot987\/Scenes-Understanding"},{"key":"573_CR35","unstructured":"Norcliffe-Brown W, Vafeais E, Parisot S. Learning conditioned graph structures for interpretable visual question answering. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R. editors. Advances in neural information processing systems, vol. 31. Montreal, Canada. 2018."},{"issue":"3","key":"573_CR36","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1037\/0022-3514.65.3.429","volume":"65","author":"SJ Read","year":"1993","unstructured":"Read SJ, Marcus-Newhall A. Explanatory coherence in social explanations: a parallel distributed processing account. Pers Soc Psychol. 1993;65(3):429.","journal-title":"Pers Soc Psychol."},{"key":"573_CR37","doi-asserted-by":"crossref","unstructured":"Ribeiro M, Singh S, Guestrin C. Why should I trust you? Explaining the predictions of any classifier. In: International conference on knowledge discovery and data mining, pp. 1135\u20131144. 2016.","DOI":"10.1145\/2939672.2939778"},{"key":"573_CR38","first-page":"1","volume":"1","author":"W Samek","year":"2017","unstructured":"Samek W, Wiegand T, M\u00fcller KR. Explainable artificial intelligence: understanding, visualizing and interpreting deep learning Models. ITU J ICT Discov Impact Artif Intell Commun Netw Serv. 2017;1:1\u201310.","journal-title":"ITU J ICT Discov Impact Artif Intell Commun Netw Serv"},{"key":"573_CR39","unstructured":"Someya Y. Lemma list for English language. 1998."},{"key":"573_CR40","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1613\/jair.1.11524","volume":"65","author":"M Sridharan","year":"2019","unstructured":"Sridharan M, Gelfond M, Zhang S, Wyatt J. REBA: a refinement-based architecture for knowledge representation and reasoning in robotics. J Artif Intell Res. 2019;65:87\u2013180.","journal-title":"J Artif Intell Res"},{"key":"573_CR41","first-page":"1","volume":"7","author":"M Sridharan","year":"2018","unstructured":"Sridharan M, Meadows B. Knowledge representation and interactive learning of domain knowledge for human-robot collaboration. Adv Cogn Syst. 2018;7:1\u201320.","journal-title":"Adv Cogn Syst."},{"issue":"4","key":"573_CR42","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1007\/s13218-019-00616-y","volume":"33","author":"M Sridharan","year":"2019","unstructured":"Sridharan M, Meadows B. Towards a theory of explanations for human-robot collaboration. Kunstliche Intelligenz. 2019;33(4):331\u201342.","journal-title":"Kunstliche Intelligenz"},{"key":"573_CR43","volume-title":"Computational models of human intelligence report 1","author":"PH Winston","year":"2018","unstructured":"Winston PH, Holmes D. The genesis enterprise: taking artificial intelligence to another level via a computational account of human story understanding. In: Computational models of human intelligence report 1. Cambridge: Massachusetts Institute of Technology; 2018."},{"key":"573_CR44","unstructured":"Yi K, Wu J, Gan C, Torralba A, Kohli P, Tenenbaum JB. Neural-symbolic VQA: disentangling reasoning from vision and language understanding. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, and Garnett R. editors. Advances in neural information processing systems. Montreal, Canada. 2018."},{"key":"573_CR45","doi-asserted-by":"crossref","unstructured":"Zhang Y, Sreedharan S, Kulkarni A, Chakraborti T, Zhuo HH, Kambhampati S. Plan explicability and predictability for robot task planning. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R editors. International conference on robotics and automation, vol. 31, pp. 313\u20131320. 2017.","DOI":"10.1109\/ICRA.2017.7989155"}],"container-title":["SN Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-021-00573-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s42979-021-00573-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-021-00573-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,25]],"date-time":"2022-12-25T22:39:13Z","timestamp":1672007953000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s42979-021-00573-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,29]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["573"],"URL":"https:\/\/doi.org\/10.1007\/s42979-021-00573-0","relation":{},"ISSN":["2662-995X","2661-8907"],"issn-type":[{"value":"2662-995X","type":"print"},{"value":"2661-8907","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,29]]},"assertion":[{"value":"14 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors (Tiago Mota, Mohan Sridharan, Ale\u0161 Leonardis) do not have any conflict of interest to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}],"article-number":"242"}}