{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,29]],"date-time":"2026-03-29T06:25:36Z","timestamp":1774765536810,"version":"3.50.1"},"reference-count":40,"publisher":"Cambridge University Press (CUP)","issue":"5","license":[{"start":{"date-parts":[[2020,9,21]],"date-time":"2020-09-21T00:00:00Z","timestamp":1600646400000},"content-version":"unspecified","delay-in-days":20,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Theory and Practice of Logic Programming"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We focus on the problem of inducing logic programs that explain models learned by the support vector machine (SVM) algorithm. The top-down sequential covering inductive logic programming (ILP) algorithms (e.g., FOIL) apply hill-climbing search using heuristics from information theory. A major issue with this class of algorithms is getting stuck in local optima. In our new approach, however, the data-dependent hill-climbing search is replaced with a model-dependent search where a globally optimal SVM model is trained first, then the algorithm looks into support vectors as the most influential data points in the model, and induces a clause that would cover the support vector and points that are most similar to that support vector. Instead of defining a fixed hypothesis search space, our algorithm makes use of SHAP, an example-specific interpreter in explainable AI, to determine a relevant set of features. This approach yields an algorithm that captures the SVM model\u2019s underlying logic and outperforms other ILP algorithms in terms of the number of induced clauses and classification evaluation metrics.<\/jats:p>","DOI":"10.1017\/s1471068420000356","type":"journal-article","created":{"date-parts":[[2020,9,21]],"date-time":"2020-09-21T05:25:42Z","timestamp":1600665942000},"page":"656-670","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":15,"title":["White-box Induction From SVM Models: Explainable AI with Logic Programming"],"prefix":"10.1017","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2460-3132","authenticated-orcid":false,"given":"FARHAD","family":"SHAKERIN","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"GOPAL","family":"GUPTA","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2020,9,21]]},"reference":[{"key":"S1471068420000356_ref4","unstructured":"4. Tianqi, Chen and Guestrin, Carlos . 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD (San Francisco, California, USA) (KDD \u201916). 785\u2013794."},{"key":"S1471068420000356_ref12","unstructured":"12. Glenn, Fung , Sathyakama, Sandilya , and Bharat Rao, R. . 2005. Rule Extraction from Linear Support Vector Machines. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining (Chicago, Illinois, USA) (KDD \u201905). ACM, New York, NY, USA, 32\u201340."},{"key":"S1471068420000356_ref21","unstructured":"21. Lundberg, Scott M , Gabriel G Erion, and Su-In Lee. 2018. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888 (2018)."},{"key":"S1471068420000356_ref26","unstructured":"26. Stephen Muggleton, Luc de Raedt, David Poole, Ivan Bratko, Peter Flach, Katsumi Inoue, and Ashwin Srinivasan. 2012. ILP Turns 20. Mach. Learn. 86, 1 (Jan. 2012), 3\u201323."},{"key":"S1471068420000356_ref25","unstructured":"25. Muggleton, Stephen . 1991. Inductive Logic Programming. New Gen. Comput. 8, 4 (Feb. 1991), 295\u2013318."},{"key":"S1471068420000356_ref24","unstructured":"24. Molnar, Christoph . 2019. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. https:\/\/christophm.github.io\/interpretable-ml-book\/."},{"key":"S1471068420000356_ref13","first-page":"2","article-title":"A survey of incremental high-utility itemset mining","volume":"8","author":"Wensheng","year":"2018","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"S1471068420000356_ref27","doi-asserted-by":"publisher","DOI":"10.1007\/11563983_15"},{"key":"S1471068420000356_ref5","unstructured":"5. Cohen, William W. . 1995. Fast Effective Rule Induction. In Proceedings of the Twelfth International Conference on International Conference on Machine Learning (Tahoe City, California, USA) (ICML\u201995). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 115\u2013123."},{"key":"S1471068420000356_ref23","unstructured":"23. David Martens, Johan Huysmans, Rudy Setiono, Jan Vanthienen, and Bart Baesens. 2008. Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring. In Rule Extraction from Support Vector Machines, Joachim, Diederich (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 33\u201363."},{"key":"S1471068420000356_ref1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07821-2"},{"key":"S1471068420000356_ref10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-04921-8_1"},{"key":"S1471068420000356_ref39","unstructured":"39. Wielemaker, Jan . 2020. The SWI-Prolog System. http:\/\/www.swi-prolog.org\/"},{"key":"S1471068420000356_ref22","unstructured":"22. Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30. Curran Associates, Inc., Long Beach, CA, 4765\u20134774."},{"key":"S1471068420000356_ref36","unstructured":"36. Farhad, Shakerin , Elmer, Salazar , and Gupta, Gopal . 2017. A new algorithm to automate inductive learning of default theories. TPLP 17, 5-6 (2017), 1010\u20131026."},{"key":"S1471068420000356_ref9","unstructured":"9. Dheeru, Dua and Graff, Casey . 2017. UCI Machine Learning Repository. http:\/\/archive.ics.uci.edu\/ml"},{"key":"S1471068420000356_ref37","unstructured":"37. Srinivasan, Ashwin . 2001. The Aleph Manual. http:\/\/web.comlab.ox.ac.uk\/oucl\/research\/areas\/machlearn\/Aleph\/"},{"key":"S1471068420000356_ref28","unstructured":"28. Haydemar, Nu\u00f1ez , Cecilio, Angulo , and Catal\u00e0, Andreu . 2002. Rule extraction from SVM. In Proc. The European Symposium on Artificial Neural Networks. Bruges, Belgium, 107\u2013112."},{"key":"S1471068420000356_ref31","volume-title":"C4.5: Programs for Machine Learning","author":"Ross Quinlan","year":"1993"},{"key":"S1471068420000356_ref34","doi-asserted-by":"publisher","DOI":"10.1145\/1055686.1055687"},{"key":"S1471068420000356_ref18","unstructured":"18. Johan, Huysmans , Rudy, Setiono , Bart, Baesens , and Vanthienen, Jan . 2008. Minerva: Sequential covering for rule extraction. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 38, 2 (2008), 299\u2013309."},{"key":"S1471068420000356_ref3","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511543357"},{"key":"S1471068420000356_ref8","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-540-75390-2","volume-title":"Rule Extraction from Support Vector Machines: An Introduction","author":"Diederich","year":"2008"},{"key":"S1471068420000356_ref40","unstructured":"40. Qiang, Zeng , Patel, Jignesh M. , and Page, David . 2014. QuickFOIL: Scalable Inductive Logic Programming. Proc. VLDB Endow. 8, 3 (Nov. 2014), 197\u2013208."},{"key":"S1471068420000356_ref15","unstructured":"15. Gunning, David . 2015. Explainable Artificial Intelligence (XAI), https:\/\/www.darpa.mil\/program\/explainable-artificial-intelligence. Retrieved June 2018."},{"key":"S1471068420000356_ref30","doi-asserted-by":"publisher","DOI":"10.1007\/BF00117105"},{"key":"S1471068420000356_ref17","unstructured":"17. Johan, Huysmans , Bart, Baesens , and Vanthienen, Jan . 2006. ITER: an algorithm for predictive regression rule extraction. In International Conference on Data Warehousing and Knowledge Discovery. Springer, Krakow, Poland, 270\u2013279."},{"key":"S1471068420000356_ref20","unstructured":"20. Niels, Landwehr , Andrea, Passerini , Luc De, Raedt , and Frasconi, Paolo . 2006. kFOIL: Learning Simple Relational Kernels. In Proceedings, The Twenty-First National Conference on Artificial Intelligence and the Eighteenth Innovative Applications of Artificial Intelligence Conference, July 16-20, 2006, Boston, Massachusetts, USA. 389\u2013394."},{"key":"S1471068420000356_ref11","first-page":"83","volume-title":"Foundations of Intelligent Systems","author":"Troels","year":"2014"},{"key":"S1471068420000356_ref32","unstructured":"32. Marco Tulio, Ribeiro , Sameer, Singh , and Guestrin, Carlos . 2016. \u201cWhy Should I Trust You?\u201d: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD 2016. 1135\u20131144."},{"key":"S1471068420000356_ref6","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022627411411"},{"key":"S1471068420000356_ref2","unstructured":"2. Rakesh, Agrawal and Srikant, Ramakrishnan . 1994. Fast Algorithms for Mining Association Rules in Large Databases. In Proceedings of the 20th International Conference on Very Large Data Bases (VLDB \u201994). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 487\u2013499."},{"key":"S1471068420000356_ref29","unstructured":"29. Plotkin, G. D. . 1971. A further note on inductive generalization. In Machine Intelligence (1971), Vol. 6. Edinburgh University Press, pages 101\u2013124."},{"key":"S1471068420000356_ref19","unstructured":"19. Niels, Landwehr , Kristian, Kersting , and De Raedt, Luc . 2005. nFOIL: Integrating Nave Bayes and FOIL. In Proceedings, The Twentieth National Conference on Artificial Intelligence and the Seventeenth Innovative Applications of Artificial Intelligence Conference, July 9-13, 2005, Pittsburgh, Pennsylvania, USA. 795\u2013800."},{"key":"S1471068420000356_ref33","unstructured":"33. Riguzzi, F. . 2016. ALEPH in SWI-Prolog. https:\/\/github.com\/friguzzi\/aleph"},{"key":"S1471068420000356_ref7","unstructured":"7. Craven, Mark W. and Shavlik, Jude W. . 1995. Extracting Tree-structured Representations of Trained Networks. In Proceedings of the 8th International Conference on Neural Information Processing Systems (Denver, Colorado) (NIPS\u201995). MIT Press, Cambridge, MA, USA, 24\u201330."},{"key":"S1471068420000356_ref16","unstructured":"16. Mark, Hall , Eibe, Frank , Geoffrey, Holmes , Bernhard, Pfahringer , Peter, Reutemann , and Witten, Ian H. . 2009. The WEKA data mining software: an update. SIGKDD Explorations 11, 1 (2009), 10\u201318."},{"key":"S1471068420000356_ref14","volume-title":"Knowledge representation, reasoning, and the design of intelligent agents: The answer-set programming approach","author":"Michael","year":"2014"},{"key":"S1471068420000356_ref38","doi-asserted-by":"crossref","unstructured":"38. Tseng, Vincent S , Cheng-Wei, Wu , Philippe Fournier-Viger, and S Yu Philip. 2016. Efficient algorithms for mining top-k high utility itemsets. IEEE Trans. on Know. and Data Engg. 28, 1 (2016), 54\u201367.","DOI":"10.1109\/TKDE.2015.2458860"},{"key":"S1471068420000356_ref35","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-39197-3_11"}],"container-title":["Theory and Practice of Logic Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1471068420000356","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,14]],"date-time":"2021-01-14T08:50:17Z","timestamp":1610614217000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1471068420000356\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9]]},"references-count":40,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["S1471068420000356"],"URL":"https:\/\/doi.org\/10.1017\/s1471068420000356","relation":{},"ISSN":["1471-0684","1475-3081"],"issn-type":[{"value":"1471-0684","type":"print"},{"value":"1475-3081","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9]]},"assertion":[{"value":"\u00a9 The Author(s), 2020. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}