{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T16:59:26Z","timestamp":1783184366063,"version":"3.54.6"},"reference-count":28,"publisher":"Cambridge University Press (CUP)","issue":"4","license":[{"start":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T00:00:00Z","timestamp":1757289600000},"content-version":"unspecified","delay-in-days":69,"URL":"https:\/\/creativecommons.org\/licenses\/by-nd\/4.0\/"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Theory and Practice of Logic Programming"],"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Recentneuro-symbolic approaches have successfully extracted symbolic rule-sets from Convolutional Neural Network-based models to enhance interpretability. However, applying similar techniques to Vision Transformers (ViTs) remains challenging due to their lack of modular concept detectors and reliance on global self-attention mechanisms. We propose a framework for symbolic rule extraction from ViTs by introducing a sparse concept layer inspired by Sparse Autoencoders (SAEs). This linear layer operates on attention-weighted patch representations and learns a disentangled, binarized representation in which individual neurons activate for high-level visual concepts. To encourage interpretability, we apply a combination of L1 sparsity, entropy minimization, and supervised contrastive loss. These binarized concept activations are used as input to the FOLD-SE-M algorithm, which generates a rule-set in the form of a logic program. Our method achieves a\n                    <jats:inline-graphic xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" mime-subtype=\"png\" xlink:href=\"S1471068425100318_inline1.png\"\/>\n                    better classification accuracy than the standard ViT while enabling symbolic reasoning. Crucially, the extracted rule-set is not merely post-hoc but acts as a logic-based decision layer that operates directly on the sparse concept representations. The resulting programs are concise and semantically meaningful. This work is the first to extract executable logic programs from ViTs using sparse symbolic representations, providing a step forward in interpretable and verifiable neuro-symbolic AI.\n                  <\/jats:p>","DOI":"10.1017\/s1471068425100318","type":"journal-article","created":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T01:20:42Z","timestamp":1757294442000},"page":"722-738","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":2,"title":["Symbolic Rule Extraction From Attention-Guided Sparse Representations in Vision Transformers"],"prefix":"10.1017","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1015-0777","authenticated-orcid":false,"given":"PARTH","family":"PADALKAR","sequence":"first","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"GOPAL","family":"GUPTA","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"56","published-online":{"date-parts":[[2025,9,8]]},"reference":[{"key":"S1471068425100318_ref8","unstructured":"Joseph, S. , Suresh, P. , Goldfarb, E. , Hufe, L. , Gandelsman, Y. , Graham, R. , Bzdok, D. , Samek, W. and Richards, B. A. 2025. Steering clip\u2019s vision transformer with sparse autoencoders. arXiv:2504.08729."},{"key":"S1471068425100318_ref27","doi-asserted-by":"crossref","unstructured":"Wang, H. and Gupta, G. 2024. FOLD-SE: An efficient rule-based machine learning algorithm with scalable explainability. In Proc. PADL 2024, Vol. 14512 of LNCS, Springer, 37\u201353.","DOI":"10.1007\/978-3-031-52038-9_3"},{"key":"S1471068425100318_ref28","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2723009"},{"key":"S1471068425100318_ref17","unstructured":"Ng, A. 2011. Sparse autoencoder. CS294A Lecture Notes, 72, 1\u201319."},{"key":"S1471068425100318_ref18","doi-asserted-by":"publisher","DOI":"10.1017\/S1471068424000322"},{"key":"S1471068425100318_ref14","unstructured":"Lim, H. , Choi, J. , Choo, J. and Schneider, S. 2025. Sparse autoencoders reveal selective remapping of visual concepts during adaptation. In Proc.The Thirteenth International Conference on Learning Representations, ICLR 2025."},{"key":"S1471068425100318_ref15","doi-asserted-by":"crossref","unstructured":"Liu, Z. , Lin, Y. , Cao, Y. , Hu, H. , Wei, Y. , Zhang, Z. , Lin, S. and Guo, B. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 10012\u201310022.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"S1471068425100318_ref1","doi-asserted-by":"crossref","unstructured":"Arias, J. , Carro, M. , Salazar, E. , Marple, K. and Gupta, G. 2018. Constraint answer set programming without grounding. Theory and Practice of Logic Programming 18, 337\u2013354.","DOI":"10.1017\/S1471068418000285"},{"key":"S1471068425100318_ref13","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5280"},{"key":"S1471068425100318_ref6","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139342124"},{"key":"S1471068425100318_ref4","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-84924-4_14"},{"key":"S1471068425100318_ref20","doi-asserted-by":"crossref","unstructured":"Padalkar, P. , Wang, H. and Gupta, G. 2024a . Nesyfold: A framework for interpretable image classification. In Proc. AAAI 2024a, AAAI Press, 4378\u20134387.","DOI":"10.1609\/aaai.v38i5.28235"},{"key":"S1471068425100318_ref16","doi-asserted-by":"crossref","unstructured":"Liu, Z. , Mao, H. , Wu, C. , Feichtenhofer, C. , Darrell, T. and Xie, S. 2022. A convnet for the 2020s. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022, 11966\u201311976.","DOI":"10.1109\/CVPR52688.2022.01167"},{"key":"S1471068425100318_ref2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511543357"},{"key":"S1471068425100318_ref12","doi-asserted-by":"crossref","unstructured":"Ko, B. and Kwak, S. 2012. Survey of computer vision-based natural disaster warning systems. In Optical Engineering, 2012, Vol. 51, 070901-1--070901-11.","DOI":"10.1117\/1.OE.51.7.070901"},{"key":"S1471068425100318_ref23","doi-asserted-by":"publisher","DOI":"10.1117\/12.2216307"},{"key":"S1471068425100318_ref24","unstructured":"Touvron, H. , Cord, M. , Douze, M. , Massa, F. , Sablayrolles, A. and J\u00e9gou, H. 2021. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, PMLR, 10347\u201310357."},{"key":"S1471068425100318_ref26","unstructured":"Vaswani, A. , Shazeer, N. , Parmar, N. , Uszkoreit, J. , Jones, L. , Gomez, A. N. , Kaiser, L.u and Polosukhin, I. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Guyon, I. , Luxburg, U. V. , Bengio, S. , Wallach, H. , Fergus, R. , Vishwanathan, S. and Garnett, R. , Eds. Vol. 30, 5998\u20136008."},{"key":"S1471068425100318_ref11","unstructured":"Kim, S. , Nam, J. and Ko, B. C. 2022. Vit-net: Interpretable vision transformers with neural tree decoder. In International conference on machine learning, PMLR, 11162\u201311172."},{"key":"S1471068425100318_ref19","doi-asserted-by":"crossref","unstructured":"Padalkar, P. , Wang, H. and Gupta, G. 2024b. Using logic programming and kernel-grouping for improving interpretability of convolutional neural networks. In Proc. PADL 2024b, Vol.\u00a014512 of LNCS, Springer, 134\u2013150.","DOI":"10.1007\/978-3-031-52038-9_9"},{"key":"S1471068425100318_ref21","unstructured":"Paul, D. , Chowdhury, A. , Xiong, X. , Chang, F.-J. , Carlyn, D. , Stevens, S. , Provost, K. , Karpatne, A. , Carstens, B. , Rubenstein, D. I. , Stewart, C. V. , Berger-Wolf, T.\u00a0Y. , Su, Y. and Chao, W.-L. 2024. A simple interpretable transformer for fine-grained image classification and analysis. arXiv:2311.04157."},{"key":"S1471068425100318_ref3","unstructured":"Bousselham, W. , Boggust, A. , Chaybouti, S. , Strobelt, H. and Kuehne, H. 2024. Legrad: An explainability method for vision transformers via feature formation sensitivity. arXiv:2404.03214."},{"key":"S1471068425100318_ref7","unstructured":"Geng, C. , Jiang, Y. , Zhao, Z. , Ye, H. , Wang, Z. and Si, X. 2025. Learning interpretable logic rules from deep vision models. arXiv:2503.10547."},{"key":"S1471068425100318_ref22","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2012.02.016"},{"key":"S1471068425100318_ref9","first-page":"1011","article-title":"Deep learning using computer vision in self driving cars for lane and traffic sign detection","volume":"12","author":"Kanagaraj","year":"2021","journal-title":"International Journal of System Assurance Engineering and Management"},{"key":"S1471068425100318_ref10","unstructured":"Khosla, P. , Teterwak, P. , Wang, C. , Sarna, A. , Tian, Y. , Isola, P. , Maschinot, A. , Liu, C. and Krishnan, D. 2020. Supervised contrastive learning. In Proc. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020. https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/d89a66c7c80a29b1bdbab0f2a1a94af8-Abstract.html."},{"key":"S1471068425100318_ref5","unstructured":"Dosovitskiy, A. , Beyer, L. , Kolesnikov, A. , Weissenborn, D. , Zhai, X. , Unterthiner, T. , Dehghani, M. , Minderer, M. , Heigold, G. , Gelly, S. , Uszkoreit, J. and Houlsby, N. 2021. An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. In Proc. 9th International Conference on Learning Representations, Virtual Event, Austria, May 3-7, 2021. https:\/\/openreview.net\/forum?id=YicbFdNTTy."},{"key":"S1471068425100318_ref25","first-page":"206","volume-title":"Computer Vision \u2013 ACCV 2021","author":"Townsend","year":"2021"}],"container-title":["Theory and Practice of Logic Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1471068425100318","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T12:56:28Z","timestamp":1761310588000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1471068425100318\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7]]},"references-count":28,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["S1471068425100318"],"URL":"https:\/\/doi.org\/10.1017\/s1471068425100318","relation":{},"ISSN":["1471-0684","1475-3081"],"issn-type":[{"value":"1471-0684","type":"print"},{"value":"1475-3081","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7]]},"assertion":[{"value":"\u00a9 The Author(s), 2025. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NoDerivatives licence (https:\/\/creativecommons.org\/licenses\/by-nd\/4.0\/), which permits re-use, distribution, and reproduction in any medium, provided that no alterations are made and the original article is properly cited.","name":"license","label":"License","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}]}}