{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T19:27:37Z","timestamp":1791314857736,"version":"4.3.1"},"reference-count":72,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T00:00:00Z","timestamp":1623974400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T00:00:00Z","timestamp":1623974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003245","name":"Ministerie van Onderwijs, Cultuur en Wetenschap","doi-asserted-by":"publisher","award":["024.004.022"],"award-info":[{"award-number":["024.004.022"]}],"id":[{"id":"10.13039\/501100003245","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    The unification of statistical (data-driven) and symbolic (knowledge-driven) methods is widely recognized as one of the key challenges of modern AI. Recent years have seen a large number of publications on such hybrid neuro-symbolic AI systems. That rapidly growing literature is highly diverse, mostly empirical, and is lacking a unifying view of the large variety of these hybrid systems. In this paper, we analyze a large body of recent literature and we propose a set of\n                    <jats:italic>modular design patterns<\/jats:italic>\n                    for such hybrid, neuro-symbolic systems. We are able to describe the architecture of a very large number of hybrid systems by composing only a small set of elementary patterns as building blocks. The main contributions of this paper are: 1) a taxonomically organised vocabulary to describe both processes and data structures used in hybrid systems; 2) a set of 15+ design patterns for hybrid AI systems organized in a set of elementary patterns and a set of compositional patterns; 3) an application of these design patterns in two realistic use-cases for hybrid AI systems. Our patterns reveal similarities between systems that were not recognized until now. Finally, our design patterns extend and refine Kautz\u2019s earlier attempt at categorizing neuro-symbolic architectures.\n                  <\/jats:p>","DOI":"10.1007\/s10489-021-02394-3","type":"journal-article","created":{"date-parts":[[2021,6,17]],"date-time":"2021-06-17T22:02:42Z","timestamp":1623967362000},"page":"6528-6546","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":98,"title":["Modular design patterns for hybrid learning and reasoning systems"],"prefix":"10.1007","volume":"51","author":[{"given":"Michael","family":"van Bekkum","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2775-8351","authenticated-orcid":false,"given":"Maaike","family":"de Boer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7913-0048","authenticated-orcid":false,"given":"Frank","family":"van Harmelen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5242-1443","authenticated-orcid":false,"given":"Andr\u00e9","family":"Meyer-Vitali","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9771-8822","authenticated-orcid":false,"given":"Annette ten","family":"Teije","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,6,18]]},"reference":[{"key":"2394_CR1","unstructured":"Arora S, Bedathur S, Ramanath M, Sharma D (2020) Iterefine: Iterative KG refinement embeddings using symbolic knowledge. CoRR, arXiv:2006.04509"},{"key":"2394_CR2","unstructured":"Asai M (2019) Unsupervised grounding of plannable first-order logic representation from images. In: Benton J, Lipovetzky N, Onaindia E, Smith DE, Srivastava S (eds) Proceedings of the Twenty-Ninth International Conference on Automated Planning and Scheduling, ICAPS 2018. AAAI Press, Berkeley, pp 583\u2013591"},{"key":"2394_CR3","doi-asserted-by":"crossref","unstructured":"Asim MN, Wasim M, Khan MUG, Mahmood W, Abbasi HM (2018) A survey of ontology learning techniques and applications. Database 2018","DOI":"10.1093\/database\/bay101"},{"key":"2394_CR4","first-page":"109:1","volume":"18","author":"SH Bach","year":"2017","unstructured":"Bach SH, Broecheler M, Huang B, Getoor L (2017) Hinge-loss markov random fields and probabilistic soft logic. J Mach Learn Res 18:109:1\u2013109:67","journal-title":"J Mach Learn Res"},{"key":"2394_CR5","doi-asserted-by":"crossref","unstructured":"Baier S, Ma Y, Tresp V (2017) Improving visual relationship detection using semantic modeling of scene descriptions. In: d\u2019Amato C, Fern\u00e1ndez M, Tamma V. A. M, L\u00e9cu\u00e9 F, Cudr\u00e9-Mauroux P, Sequeda J. F, Lange C, Heflin J (eds) The semantic web - ISWC 2017 - 16th International semantic web conference, proceedings, Part I, volume 10587 of Lecture notes in computer science. Springer, Vienna, pp 53\u201368","DOI":"10.1007\/978-3-319-68288-4_4"},{"key":"2394_CR6","unstructured":"Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi VF, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, G\u00fcl\u00e7ehre \u00c7, Song F, Ballard AJ, Gilmer J, Dahl GE, Vaswani A, Allen K, Nash C, Langston V, Dyer C, Heess N, Wierstra D, Kohli P, Botvinick M, Vinyals O, Li Y, Pascanu R (2018) Relational inductive biases, deep learning, and graph networks. CoRR, arXiv:1806.01261"},{"key":"2394_CR7","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1007\/s11023-007-9086-y","volume":"18","author":"I Berkeley","year":"2008","unstructured":"Berkeley I (2008) What the <\u20090.70, 1.17, 0.99, 1.07 > is a symbol?. Minds Mach 18:93\u2013105","journal-title":"Minds Mach"},{"key":"2394_CR8","doi-asserted-by":"crossref","unstructured":"Bian J, Gao B, Liu T (2014) Knowledge-powered deep learning for word embedding. In: Calders T, Esposito F, H\u00fcllermeier E, Meo R (eds) Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2014, Nancy, France, September 15\u201319, 2014. Proceedings, Part I, volume 8724 of Lecture Notes in Computer Science, pp 132\u2013148","DOI":"10.1007\/978-3-662-44848-9_9"},{"key":"2394_CR9","doi-asserted-by":"crossref","unstructured":"Borgwardt K, Ghisu E, Llinares-L\u00f3pez F, O\u2019Bray L, Rieck B (2020) Graph kernels: State-of-the-art and future challenges","DOI":"10.1561\/9781680837711"},{"key":"2394_CR10","doi-asserted-by":"crossref","unstructured":"Bouraoui Z, Jameel S, Schockaert S (2017) Inductive reasoning about ontologies using conceptual spaces. In: Singh SP, Markovitch S (eds) Proceedings of the Thirty-First AAAI conference on artificial intelligence. AAAI Press, San Francisco, pp 4364\u20134370","DOI":"10.1609\/aaai.v31i1.11162"},{"key":"2394_CR11","unstructured":"Brewster CA (2008) Mind the gap : bridging from text to ontological knowledge. PhD thesis, University of Sheffield, UK"},{"key":"2394_CR12","doi-asserted-by":"crossref","unstructured":"Cimiano P, M\u00e4dche A, Staab S, V\u00f6lker J (2009) Ontology learning. In: Handbook on ontologies. Springer, 245\u2013267","DOI":"10.1007\/978-3-540-92673-3_11"},{"key":"#cr-split#-2394_CR13.1","doi-asserted-by":"crossref","unstructured":"Ciravegna G, Giannini F, Gori M, Maggini M, Melacci S (2020) Human-driven FOL explanations of deep learning. In: Bessiere C","DOI":"10.24963\/ijcai.2020\/309"},{"key":"#cr-split#-2394_CR13.2","unstructured":"(ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020. ijcai.org, pp 2234-2240"},{"key":"2394_CR14","unstructured":"Cohen WW, Sun H, Hofer RA, Siegler M (2020) Scalable neural methods for reasoning with a symbolic knowledge base. arXiv:2002.06115"},{"key":"2394_CR15","doi-asserted-by":"crossref","unstructured":"Costantini S (2002) Meta-reasoning: A survey. In: Kakas AC, Sadri F (eds) Computational Logic: Logic Programming and Beyond, Essays in Honour of Robert A. Kowalski, Part II, volume 2408 of Lecture Notes in Computer Science. Springer, pp 253\u2013288","DOI":"10.1007\/3-540-45632-5_11"},{"issue":"10","key":"2394_CR16","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/3271625","volume":"61","author":"A Darwiche","year":"2018","unstructured":"Darwiche A (2018) Human-level intelligence or animal-like abilities? Commun ACM 61(10):56\u201367","journal-title":"Commun ACM"},{"key":"2394_CR17","unstructured":"d\u2019Avila Garcez A, Lamb LC (2020) Neurosymbolic AI: The 3rd wave"},{"issue":"3 and 4","key":"2394_CR18","first-page":"300","volume":"12","author":"MH de Boer","year":"2019","unstructured":"de Boer MH, Verhoosel JP (2019) Creating and evaluating data-driven ontologies. Int J Adv Softw 12(3 and 4):300\u2013309","journal-title":"Int J Adv Softw"},{"key":"2394_CR19","doi-asserted-by":"crossref","unstructured":"Demeester T, Rockt\u00e4schel T, Riedel S (2016) Lifted rule injection for relation embeddings. In: Su J, Carreras X, Duh K (eds) Proceedings of the 2016 conference on empirical methods in natural language processing, EMNLP. The Association for Computational Linguistics, Austin, pp 1389\u20131399","DOI":"10.18653\/v1\/D16-1146"},{"key":"2394_CR20","unstructured":"Detassis F, Lombardi M, Milano M (2020) Teaching the old dog new tricks Supervised learning with constraints. CoRR, arXiv:2002.10766"},{"key":"#cr-split#-2394_CR21.1","doi-asserted-by":"crossref","unstructured":"Donadello I, Serafini L, d'Avila Garcez AS (2017) Logic tensor networks for semantic image interpretation. In: Sierra C","DOI":"10.24963\/ijcai.2017\/221"},{"key":"#cr-split#-2394_CR21.2","unstructured":"(ed) Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017. ijcai.org, Melbourne, pp 1596-1602"},{"key":"2394_CR22","unstructured":"Ebrahimi M, Sarker MK, Bianchi F, Xie N, Doran D, Hitzler P (2018) Reasoning over RDF knowledge bases using deep learning. CoRR, arXiv:1811.04132"},{"key":"2394_CR23","unstructured":"Fang M, Zhou T, Du Y, Han L, Zhang Z (2019) Curriculum-guided hindsight experience replay. In: Wallach HM, Larochelle H, Beygelzimer A, d\u2019Alch\u00e9-Buc F, Fox E. B., Garnett R. (eds) Advances in neural information processing systems 32: Annual conference on neural information processing systems 2019, NeurIPS 2019, Vancouver, pp 12602\u201312613"},{"key":"2394_CR24","unstructured":"Fournier P, Sigaud O, Chetouani M, Oudeyer P (2018) Accuracy-based curriculum learning in deep reinforcement learning. CoRR, arXiv:1806.09614"},{"key":"2394_CR25","volume-title":"Design patterns: elements of reusable object-oriented software","author":"E Gamma","year":"1994","unstructured":"Gamma E, Helm R, Johnson R, Vlissides JM (1994) Design patterns: elements of reusable object-oriented software, 1st edn. Addison-Wesley Professional, Boston","edition":"1st edn."},{"key":"2394_CR26","unstructured":"Garcez A. d., Gori M, Lamb LC, Serafini L, Spranger M, Tran SN (2019) Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv:1905.06088"},{"key":"2394_CR27","unstructured":"Garnelo M, Arulkumaran K, Shanahan M (2016) Towards deep symbolic reinforcement learning. CoRR, arXiv:1609.05518"},{"key":"2394_CR28","doi-asserted-by":"crossref","unstructured":"Geibel P (2006) Reinforcement learning for mdps with constraints. In: F\u00fcrnkranz J, Scheffer T, Spiliopoulou M (eds) Machine Learning: ECML 2006, 17th European conference on machine learning, 2006, Proceedings, volume 4212 of Lecture Notes in Computer Science. Springer, Berlin, pp 646\u2013653","DOI":"10.1007\/11871842_63"},{"key":"2394_CR29","doi-asserted-by":"crossref","unstructured":"Getoor L (2013) Probabilistic soft logic: A scalable approach for markov random fields over continuous-valued variables - (abstract of keynote talk). In: Morgenstern L, Stefaneas PS, L\u00e9vy F, Wyner AZ, Paschke A (eds) Theory, practice, and applications of rules on the Web - 7th International Symposium, RuleML 2013, Proceedings, volume 8035 of Lecture Notes in Computer Science. Springer, Seattle, p 1","DOI":"10.1007\/978-3-642-39617-5_1"},{"key":"2394_CR30","unstructured":"Hohenecker P, Lukasiewicz T (2017) Deep learning for ontology reasoning. CoRR, arXiv:1705.10342"},{"key":"2394_CR31","doi-asserted-by":"crossref","unstructured":"Hutter F., Kotthoff L., Vanschoren J. (eds) (2019) Automatic machine learning: methods, systems, challenges. Springer , Berlin","DOI":"10.1007\/978-3-030-05318-5"},{"key":"2394_CR32","unstructured":"Icarte RT, Klassen TQ, Valenzano RA, McIlraith SA (2018) Teaching multiple tasks to an RL agent using LTL. In: Andr\u00e9 E, Koenig S, Dastani M, Sukthankar G (eds) Proceedings of the 17th international conference on autonomous agents and multiagent systems, AAMAS 2018, International foundation for autonomous agents and multiagent systems. ACM, Stockholm, pp 452\u2013461"},{"key":"2394_CR33","doi-asserted-by":"crossref","unstructured":"Illanes L, Yan X, Icarte RT, McIlraith. SA (2020) Symbolic plans as high-level instructions for reinforcement learning. In: Beck JC, Buffet O, Hoffmann J, Karpas E, Sohrabi S (eds) Proceedings of the thirtieth international conference on automated planning and scheduling. AAAI Press, Nancy, pp 540\u2013550","DOI":"10.1609\/icaps.v30i1.6750"},{"key":"2394_CR34","doi-asserted-by":"crossref","unstructured":"Inoue K, Ohwada H, Yamamoto A (2017) Special issue on inductive logic programming","DOI":"10.1007\/978-3-319-40566-7"},{"key":"2394_CR35","doi-asserted-by":"crossref","unstructured":"Konstantopoulos S, Charalambidis A (2010) Formulating description logic learning as an inductive logic programming task. In: FUZZ-IEEE 2010, IEEE International Conference on Fuzzy Systems, 2010, Proceedings. IEEE, Barcelona, pp 1\u20137","DOI":"10.1109\/FUZZY.2010.5584417"},{"key":"2394_CR36","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/j.compbiomed.2016.06.019","volume":"76","author":"R Kop","year":"2016","unstructured":"Kop R, et al. (2016) And Predictive modeling of colorectal cancer using a dedicated pre-processing pipeline on routine electronic medical records. Comp Bio Med 76:30\u201338","journal-title":"Comp Bio Med"},{"issue":"11","key":"2394_CR37","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y Lecun","year":"1998","unstructured":"Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278\u20132324","journal-title":"Proc IEEE"},{"key":"2394_CR38","unstructured":"Liello LD, Ardino P, Gobbi J, Morettin P, Teso S, Passerini A (2020) Efficient generation of structured objects with constrained adversarial networks. CoRR, arXiv:2007.13197"},{"key":"2394_CR39","unstructured":"Loos SM, Irving G, Szegedy C, Kaliszyk C (2017) Deep network guided proof search. In: Eiter T, Sands D (eds) LPAR-21, 21st International conference on logic for programming, artificial intelligence and reasoning, volume 46 of EPiC Series in Computing. EasyChair, Maun, pp 85\u2013105"},{"key":"2394_CR40","unstructured":"Manhaeve R, Dumancic S, Kimmig A, Demeester T, Raedt LD (2018) Deepproblog: Neural probabilistic logic programming. In: Bengio S, Wallach HM, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31: Annual conference on neural information processing systems 2018, NeurIPS 2018, Montr\u00e9al, pp 3753\u20133763"},{"key":"2394_CR41","unstructured":"Marra G, Diligenti M, Giannini F, Gori M, Maggini M (2020) Relational neural machines. In: Giacomo G. D., Catal\u00e1 A, Dilkina B, Milano M, Barro S, Bugar\u00edn A, Lang J (eds) ECAI 2020 - 24th European conference on artificial intelligence, 2020 - Including 10th conference on prestigious applications of artificial intelligence (PAIS 2020), volume 325 of Frontiers in artificial intelligence and Applications. IOS Press, Santiago de Compostela, pp 1340\u20131347"},{"key":"2394_CR42","doi-asserted-by":"crossref","unstructured":"Marra G, Giannini F, Diligenti M, Gori M (2019) LYRICS: A general interface layer to integrate logic inference and deep learning. In: Brefeld U, Fromont \u00c9, Hotho A, Knobbe AJ, Maathuis MH, Robardet C (eds) Machine learning and knowledge discovery in databases - European conference, ECML PKDD 2019, Proceedings, Part II, volume 11907 of Lecture notes in computer science. Springer, W\u00fcrzburg, pp 283\u2013298","DOI":"10.1007\/978-3-030-46147-8_17"},{"key":"2394_CR43","doi-asserted-by":"crossref","unstructured":"Meilicke C, Chekol MW, Ruffinelli D, Stuckenschmidt H (2019) Anytime bottom-up rule learning for knowledge graph completion. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, pp 3137\u20133143","DOI":"10.24963\/ijcai.2019\/435"},{"issue":"1","key":"2394_CR44","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1109\/JPROC.2015.2483592","volume":"104","author":"M Nickel","year":"2016","unstructured":"Nickel M, Murphy K, Tresp V, Gabrilovich E (2016) A review of relational machine learning for knowledge graphs. Proc IEEE 104(1):11\u201333","journal-title":"Proc IEEE"},{"key":"2394_CR45","doi-asserted-by":"crossref","unstructured":"Padia A, Martin D, Patel-Schneider PF (2018) Automating class\/instance representational choices in knowledge bases. In: Faron-Zucker C, Ghidini C, Napoli A, Toussaint Y (eds) Knowledge engineering and knowledge management - 21st International conference, EKAW 2018, 2018 Proceedings, volume 11313 of Lecture Notes in Computer Science. Springer, pp 273\u2013288","DOI":"10.1007\/978-3-030-03667-6_18"},{"issue":"3","key":"2394_CR46","doi-asserted-by":"publisher","first-page":"489","DOI":"10.3233\/SW-160218","volume":"8","author":"H Paulheim","year":"2017","unstructured":"Paulheim H (2017) Knowledge graph refinement: A survey of approaches and evaluation methods. Semant Web 8(3):489\u2013508","journal-title":"Semant Web"},{"key":"2394_CR47","doi-asserted-by":"crossref","unstructured":"Pearl J (2018) Theoretical impediments to machine learning with seven sparks from the causal revolution. In: Chang Y, Zhai C, Liu Y, Maarek Y (eds) Proceedings of the Eleventh ACM International conference on web search and data mining, WSDM 2018. ACM, Marina Del Rey, p 3","DOI":"10.1145\/3159652.3176182"},{"key":"2394_CR48","unstructured":"Perlis D, Brody J, Kraus S, Miller MJ (2017) The internal reasoning of robots. In: Gordon AS, Miller R, Tura\u0307n G (eds) Proceedings of the Thirteenth International Symposium on Commonsense Reasoning, COMMONSENSE, volume 2052 of CEUR Workshop Proceedings. CEUR-WS.org, London, p 2017"},{"key":"#cr-split#-2394_CR49.1","doi-asserted-by":"crossref","unstructured":"Raedt LD, Dumancic S., Manhaeve R, Marra G (2020) From statistical relational to neuro-symbolic artificial intelligence. In: Bessiere C","DOI":"10.24963\/ijcai.2020\/688"},{"key":"#cr-split#-2394_CR49.2","unstructured":"(ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020. ijcai.org, pp 4943-4950"},{"issue":"1-2","key":"2394_CR50","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1007\/s10994-006-5833-1","volume":"62","author":"M Richardson","year":"2006","unstructured":"Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1-2):107\u2013136","journal-title":"Mach Learn"},{"key":"2394_CR51","unstructured":"Rockt\u00e4schel T, Riedel S, Vishwanathan SVN (2017) End-to-end differentiable proving. In: Guyon II, von Luxburg U, Bengio S, Wallach HM, Fergus R, Garnett R (eds) Advances in neural information processing systems 30: Annual conference on neural information processing systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp 3791\u20133803"},{"key":"2394_CR52","unstructured":"Sala F, Sa CD, Gu A, R\u00e9 C (2018) Representation tradeoffs for hyperbolic embeddings. In: Dy JG, Krause A (eds) Proceedings of the 35th international conference on machine learning, ICML 2018, volume 80 of Proceedings of machine learning research. PMLR, Stockholmsm\u00e1ssan, pp 4457\u20134466"},{"key":"2394_CR53","unstructured":"Sarker MK, Xie N, Doran D, Raymer M, Hitzler P (2017) Explaining trained neural networks with semantic web technologies: First steps. In: Besold TR, d\u2019Avila Garcez AS, Noble I (eds) Proceedings of the twelfth international workshop on neural-symbolic learning and reasoning, NeSy 2017, volume 2003 of CEUR workshop proceedings. CEUR-WS.org, London"},{"key":"2394_CR54","unstructured":"Serafini L, Garcez A. d. (2016) Logic tensor networks: Deep learning and logical reasoning from data and knowledge. arXiv:1606.04422"},{"key":"2394_CR55","unstructured":"Shang Z, Zgraggen E, Kraska T (2109) Alpine meadow: A system for interactive automl. In: NEURIPS"},{"issue":"7676","key":"2394_CR56","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","volume":"550","author":"D Silver","year":"2017","unstructured":"Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A, Chen Y, Lillicrap TP, Hui F, Sifre L, van den Driessche G, Graepel T, Hassabis D (2017) Mastering the game of go without human knowledge. Nat 550(7676):354\u2013359","journal-title":"Nat"},{"key":"2394_CR57","doi-asserted-by":"crossref","unstructured":"Silvestri M, Lombardi M, Milano M (2020) Injecting domain knowledge in neural networks: a controlled experiment on a constrained problem. CoRR, arXiv:2002.10742","DOI":"10.1007\/978-3-030-78230-6_17"},{"key":"#cr-split#-2394_CR58.1","doi-asserted-by":"crossref","unstructured":"Suttner CB, Ertel W (1990) Automatic acquisition of search guiding heuristics. In: Stickel ME","DOI":"10.1007\/3-540-52885-7_108"},{"key":"#cr-split#-2394_CR58.2","unstructured":"(ed) 10th international conference on automated deduction, Kaiserslautern, FRG, 1990 Proceedings, volume 449 of Lecture Notes in Computer Science. Springer, pp 470-484"},{"key":"2394_CR59","doi-asserted-by":"crossref","unstructured":"Tiddi I, d\u2019Aquin M, Motta E (2015) Data patterns explained with linked data. In: Bifet A, May M, Zadrozny B, Gavald\u00e0 R, Pedreschi D, Bonchi F, Cardoso JS, Spiliopoulou M (eds) Machine learning and knowledge discovery in databases - European conference, ECML PKDD 2015, 2015 Proceedings, Part III, volume 9286 of Lecture Notes in Computer Science. Springer, Porto, pp 271\u2013275","DOI":"10.1007\/978-3-319-23461-8_28"},{"issue":"1-3","key":"2394_CR60","doi-asserted-by":"publisher","first-page":"97","DOI":"10.13052\/jwe1540-9589.18133","volume":"18","author":"F van Harmelen","year":"2019","unstructured":"van Harmelen F, ten Teije A (2019) A boxology of design patterns for hybrid learning and reasoning systems. J Web Eng 18(1-3):97\u2013124","journal-title":"J Web Eng"},{"key":"2394_CR61","first-page":"19","volume":"18","author":"L Von Rueden","year":"2019","unstructured":"Von Rueden L, Mayer S, Garcke J, Bauckhage C, Schuecker J (2019) Informed machine learning\u2013towards a taxonomy of explicit integration of knowledge into machine learning. Learning 18:19\u201320","journal-title":"Learning"},{"issue":"12","key":"2394_CR62","doi-asserted-by":"publisher","first-page":"2724","DOI":"10.1109\/TKDE.2017.2754499","volume":"29","author":"Q Wang","year":"2017","unstructured":"Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: A survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724\u20132743, 12","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2394_CR63","unstructured":"Weld DS, Bansal G (2018) Intelligible artificial intelligence. CoRR, arXiv:1803.04263"},{"key":"2394_CR64","first-page":"193","volume-title":"Visualization of patterns for hybrid learning and reasoning with human involvement","author":"HF Witschel","year":"2021","unstructured":"Witschel HF, Pande C, Martin A, Laurenzi E, Hinkelmann K (2021) Visualization of patterns for hybrid learning and reasoning with human involvement. Springer International Publishing, Cham, pp 193\u2013204"},{"issue":"4","key":"2394_CR65","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2333112.2333115","volume":"44","author":"W Wong","year":"2012","unstructured":"Wong W, Liu W, Bennamoun M (2012) Ontology learning from text: A look back and into the future. ACM Comput Surv (CSUR) 44(4):1\u201336","journal-title":"ACM Comput Surv (CSUR)"},{"key":"2394_CR66","unstructured":"Xu J, Zhang Z, Friedman T, Liang Y, den Broeck GV (2018) A semantic loss function for deep learning with symbolic knowledge. In: Dy JG, Krause A (eds) Proceedings of the 35th international conference on machine learning, ICML 2018, volume 80 of JMLR Workshop and conference proceedings. JMLR.org, Stockholmsma\u0307ssan, pp 5498\u20135507"},{"key":"2394_CR67","doi-asserted-by":"crossref","unstructured":"Zhang W, Paudel B, Wang L, Chen J, Zhu H, Zhang W, Bernstein A, Chen H (2019) Iteratively learning embeddings and rules for knowledge graph reasoning. In: Liu L, White RW, Mantrach A, Silvestri F, McAuley JJ, Baeza-Yates R, Zia L (eds) The World Wide Web Conference, WWW 2019. ACM, San Francisco, pp 2366\u20132377","DOI":"10.1145\/3308558.3313612"},{"key":"2394_CR68","unstructured":"Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2018) Graph neural networks: A review of methods and applications. arXiv:1812.08434"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-021-02394-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-021-02394-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-021-02394-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,31]],"date-time":"2022-12-31T10:10:59Z","timestamp":1672481459000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-021-02394-3"}},"subtitle":["a taxonomy, patterns and use cases"],"short-title":[],"issued":{"date-parts":[[2021,6,18]]},"references-count":72,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["2394"],"URL":"https:\/\/doi.org\/10.1007\/s10489-021-02394-3","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"value":"0924-669X","type":"print"},{"value":"1573-7497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,18]]},"assertion":[{"value":"27 March 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}