{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T15:31:29Z","timestamp":1786980689482,"version":"3.56.0"},"reference-count":66,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T00:00:00Z","timestamp":1687478400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T00:00:00Z","timestamp":1687478400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001942","name":"CHIST-ERA","doi-asserted-by":"publisher","award":["CHIST-ERA-19-XAI-005"],"award-info":[{"award-number":["CHIST-ERA-19-XAI-005"]}],"id":[{"id":"10.13039\/501100001942","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Alma Mater Studiorum - Universit\u00e0 di Bologna"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Agent Multi-Agent Syst"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Bridging intelligent symbolic agents and sub-symbolic predictors is a long-standing research goal in AI. Among the recent integration efforts, symbolic knowledge injection (SKI) proposes algorithms aimed at steering sub-symbolic predictors\u2019 learning towards compliance w.r.t. pre-existing symbolic knowledge bases. However, state-of-the-art contributions about SKI mostly tackle injection from a foundational perspective, often focussing solely on improving the predictive performance of the sub-symbolic predictors undergoing injection. Technical contributions, in turn, are tailored on individual methods\/experiments and therefore poorly interoperable with agent technologies as well as among each others. Intelligent agents may exploit SKI to serve many purposes other than predictive performance alone\u2014provided that, of course, adequate technological support exists: for instance, SKI may allow agents to tune computational, energetic, or data requirements of sub-symbolic predictors. Given that different algorithms may exist to serve all those many purposes, some criteria for\n                    <jats:italic>algorithm selection<\/jats:italic>\n                    as well as a suitable\n                    <jats:italic>technology<\/jats:italic>\n                    should be available to let agents dynamically select and exploit the most suitable algorithm for the problem at hand. Along this line, in this work we design a set of\n                    <jats:italic>quality-of-service<\/jats:italic>\n                    (QoS)\n                    <jats:italic>metrics<\/jats:italic>\n                    for SKI, and a\n                    <jats:italic>general-purpose software API<\/jats:italic>\n                    to enable their application to various SKI algorithms\u2014namely, platform for symbolic knowledge injection (PSyKI). We provide an abstract formulation of four QoS metrics for SKI, and describe the design of PSyKI according to a software engineering perspective. Then we discuss how our QoS metrics are supported by PSyKI. Finally, we demonstrate the effectiveness of both our QoS metrics and PSyKI via a number of experiments, where SKI is both applied and assessed via our proposed API. Our empirical analysis demonstrates both the soundness of our proposed metrics and the versatility of PSyKI as the first software tool supporting the application, interchange, and numerical assessment of SKI techniques. To the best of our knowledge, our proposals represent the first attempt to introduce QoS metrics for SKI, and the software tools enabling their\n                    <jats:italic>practical<\/jats:italic>\n                    exploitation for both human and computational agents. In particular, our contributions could be exploited to automate and\/or compare the manifold SKI algorithms from the state of the art. Hence moving a concrete step forward the engineering of efficient, robust, and trustworthy software applications that integrate symbolic agents and sub-symbolic predictors.\n                  <\/jats:p>","DOI":"10.1007\/s10458-023-09609-6","type":"journal-article","created":{"date-parts":[[2023,6,24]],"date-time":"2023-06-24T05:37:22Z","timestamp":1687585042000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Symbolic knowledge injection meets intelligent agents: QoS metrics and experiments"],"prefix":"10.1007","volume":"37","author":[{"given":"Andrea","family":"Agiollo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Rafanelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matteo","family":"Magnini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giovanni","family":"Ciatto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Omicini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,6,23]]},"reference":[{"issue":"2","key":"9609_CR1","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1109\/TNNLS.2020.2979670","volume":"32","author":"DW Otter","year":"2021","unstructured":"Otter, D. W., Medina, J. R., & Kalita, J. K. (2021). A survey of the usages of deep learning for natural language processing. IEEE Transactions on Neural Networks and Learning Systems, 32(2), 604\u2013624. https:\/\/doi.org\/10.1109\/TNNLS.2020.2979670","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"9609_CR2","doi-asserted-by":"publisher","first-page":"19143","DOI":"10.1109\/ACCESS.2019.2896880","volume":"7","author":"AB Nassif","year":"2019","unstructured":"Nassif, A. B., Shahin, I., Attili, I. B., Azzeh, M., & Shaalan, K. (2019). Speech recognition using deep neural networks: A systematic review. IEEE Access, 7, 19143\u201319165. https:\/\/doi.org\/10.1109\/ACCESS.2019.2896880","journal-title":"IEEE Access"},{"issue":"11","key":"9609_CR3","doi-asserted-by":"publisher","first-page":"3212","DOI":"10.1109\/TNNLS.2018.2876865","volume":"30","author":"Z-Q Zhao","year":"2019","unstructured":"Zhao, Z.-Q., Zheng, P., Xu, S.-T., & Wu, X. (2019). Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems, 30(11), 3212\u20133232. https:\/\/doi.org\/10.1109\/TNNLS.2018.2876865","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"9609_CR4","unstructured":"Agiollo, A., & Omicini, A. (2022). GNN2GNN: Graph neural networks to generate neural networks. In J. Cussens & K. Zhang (Eds.), Uncertainty in artificial intelligence. Proceedings of machine learning research, vol. 180, pp. 32\u201342. ML Research Press, Maastricht, The Netherlands. Proceedings of the thirty-eighth conference on uncertainty in artificial intelligence, UAI 2022, 1\u20135 August 2022, Eindhoven, The Netherlands. https:\/\/proceedings.mlr.press\/v180\/agiollo22a.html"},{"issue":"1","key":"9609_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-00376-2","volume":"4","author":"A Esteva","year":"2021","unstructured":"Esteva, A., Chou, K., Yeung, S., Naik, N., Madani, A., Mottaghi, A., Liu, Y., Topol, E., Dean, J., & Socher, R. (2021). Deep learning-enabled medical computer vision. NPJ Digital Medicine, 4(1), 1\u20139. https:\/\/doi.org\/10.1038\/s41746-020-00376-2","journal-title":"NPJ Digital Medicine"},{"issue":"3","key":"9609_CR6","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1002\/rob.21918","volume":"37","author":"SM Grigorescu","year":"2020","unstructured":"Grigorescu, S. M., Trasnea, B., Cocias, T. T., & Macesanu, G. (2020). A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3), 362\u2013386. https:\/\/doi.org\/10.1002\/rob.21918","journal-title":"Journal of Field Robotics"},{"key":"9609_CR7","doi-asserted-by":"publisher","unstructured":"Calegari, R., Ciatto, G., Mascardi, V., & Omicini, A. (2021). Logic-based technologies for multi-agent systems: A systematic literature review. Autonomous Agents and Multi-Agent Systems, 35(1), 1\u20131167. https:\/\/doi.org\/10.1007\/s10458-020-09478-3. Collection \u201cCurrent Trends in Research on Software Agents and Agent-Based Software Development\u201d.","DOI":"10.1007\/s10458-020-09478-3"},{"key":"9609_CR8","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45628-7","volume-title":"Computational logic: Logic programming and beyond, essays in honour of Robert A. Kowalski, part I. Lecture Notes in Computer Science","year":"2002","unstructured":"Kakas, A. C., & Sadri, F. (Eds.). (2002). Computational logic: Logic programming and beyond, essays in honour of Robert A. Kowalski, part I. Lecture Notes in Computer Science (Vol. 2407). New York: Springer. https:\/\/doi.org\/10.1007\/3-540-45628-7"},{"key":"9609_CR9","doi-asserted-by":"publisher","unstructured":"Diligenti, M., Roychowdhury, S., & Gori, M. (2017) Integrating prior knowledge into deep learning. In 2017 16th IEEE international conference on machine learning and applications (ICMLA) (pp. 920\u2013923). https:\/\/doi.org\/10.1109\/ICMLA.2017.00-37","DOI":"10.1109\/ICMLA.2017.00-37"},{"key":"9609_CR10","unstructured":"Xu, J., Zhang, Z., Friedman, T., Liang, Y., & den Broeck, G. V. (2018). A semantic loss function for deep learning with symbolic knowledge. In: Dy,\nJ., Krause, A. (Eds.), 35th International Conference on Machine Learning\n(ICML 2018).Proceedings of Machine Learning Research (PLMR),\nvol. 80, pp. 5502\u20135511. Stockholmsm\u00e4ssan, Stockholm, Sweden. https:\/\/proceedings.mlr.press\/v80\/xu18h.html"},{"key":"9609_CR11","doi-asserted-by":"publisher","unstructured":"Magnini, M., Ciatto, G., & Omicini, A. (2022). On the design of PSyKI: A platform for symbolic knowledge injection into sub-symbolic predictors. In D. Calvaresi, A. Najjar, M. Winikoff, & K. Fr\u00e4mling (Eds.), Explainable and transparent AI and multi-agent systems. Lecture Notes in Computer Science (Vol. 13283, pp. 90\u2013108. Springer, Cham, Switzerland. Chap. 6. 4th International Workshop, EXTRAAMAS 2022, Virtual Event, Revised Selected Papers. https:\/\/doi.org\/10.1007\/978-3-031-15565-9_6","DOI":"10.1007\/978-3-031-15565-9_6"},{"issue":"1","key":"9609_CR12","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3233\/IA-190036","volume":"14","author":"R Calegari","year":"2020","unstructured":"Calegari, R., Ciatto, G., & Omicini, A. (2020). On the integration of symbolic and sub-symbolic techniques for XAI: A survey. Intelligenza Artificiale, 14(1), 7\u201332. https:\/\/doi.org\/10.3233\/IA-190036","journal-title":"Intelligenza Artificiale"},{"key":"9609_CR13","unstructured":"Agiollo, A., Rafanelli, A., & Omicini, A. (2022). Towards quality-of-service metrics for symbolic knowledge injection. In A. Ferrando & V. Mascardi (Eds.), WOA 2022\u201423rd Workshop \u201cFrom Objects to Agents\u201d. CEUR Workshop Proceedings (Vol. 3261, pp. 30\u201347). http:\/\/ceur-ws.org\/Vol-3261\/paper3.pdf"},{"key":"9609_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-86718-7","volume-title":"First-order logic","author":"RM Smullyan","year":"1968","unstructured":"Smullyan, R. M. (1968). First-order logic. New York: Springer."},{"key":"9609_CR15","unstructured":"Besold, T. R., d\u2019Avila Garcez, A. S., Bader, S., Bowman, H., Domingos, P. M., Hitzler, P., et al. (2017). Neural-symbolic learning and reasoning: A survey and interpretation. CoRR abs\/1711.03902 arxiv:1711.03902"},{"key":"9609_CR16","unstructured":"Xie, Y., Xu, Z., Meel, K. S., Kankanhalli, M. S., & Soh, H. (2019). Embedding symbolic knowledge into deep networks. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, E. B. Fox, & R. Garnett (Eds.), Advances in neural information processing systems, 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8\u201314, 2019, Vancouver, BC, Canada, pp. 4235\u20134245. https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/7b66b4fd401a271a1c7224027ce111bc-Abstract.html"},{"key":"9609_CR17","unstructured":"Towell, G. G., Shavlik, J. W. & Noordewier, M. O. (1990). Refinement of approximate domain theories by knowledge-based neural networks. In Proceedings of the 8th national conference on artificial intelligence (pp. 861\u2013866)"},{"key":"9609_CR18","first-page":"871","volume":"5","author":"V Tresp","year":"1992","unstructured":"Tresp, V., Hollatz, J. & Ahmad, S. (1992) Network structuring and training using rule-based knowledge. Advances in Neural Information Processing Systems, 5, 871-878","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"1","key":"9609_CR19","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1023\/A:1008328630915","volume":"11","author":"AS d\u2019Avila Garcez","year":"1999","unstructured":"d\u2019Avila Garcez, A. S., & Zaverucha, G. (1999). The connectionist inductive learning and logic programming system. Applied Intelligence, 11(1), 59\u201377. https:\/\/doi.org\/10.1023\/A:1008328630915","journal-title":"Applied Intelligence"},{"key":"9609_CR20","unstructured":"d\u2019Avila Garcez, A. S., & Gabbay, D. M. (2004). Fibring neural networks. In D. L. McGuinness & G. Ferguson (Eds.), Proceedings of the nineteenth national conference on artificial intelligence, sixteenth conference on innovative applications of artificial intelligence, July 25\u201329, San Jose, California, USA (pp. 342\u2013347). AAAI Press\/The MIT Press. http:\/\/www.aaai.org\/Library\/AAAI\/2004\/aaai04-055.php"},{"key":"9609_CR21","unstructured":"Bader, S., d\u2019Avila Garcez, A. S., & Hitzler, P. (2005). Computing first-order logic programs by fibring artificial neural networks. In I. Russell, & Z. Markov (Eds.), Proceedings of the eighteenth international florida artificial intelligence research society conference (pp. 314\u2013319). Clearwater Beach, FL: AAAI Press. http:\/\/www.aaai.org\/Library\/FLAIRS\/2005\/flairs05-052.php"},{"key":"9609_CR22","unstructured":"Chang, M., Ratinov, L., & Roth, D. (2007). Guiding semi-supervision with constraint-driven learning. In J. A. Carroll, A. van\u00a0den Bosch, & A. Zaenen (Eds.), ACL 2007, proceedings of the 45th annual meeting of the association for computational linguistics, June 23\u201330, Prague, Czech Republic. https:\/\/aclanthology.org\/P07-1036\/"},{"key":"9609_CR23","unstructured":"Nickel, M., Tresp, V., Kriegel, H.-P. (2011). A three-way model for collective learning on multi-relational data. ICML, 11, 809\u2013816. https:\/\/icml.cc\/2011\/papers\/438icmlpaper.pdf"},{"key":"9609_CR24","doi-asserted-by":"crossref","unstructured":"Chang, K.-W., Yih, W.-t., Yang, B., & Meek, C. (2014). Typed tensor decomposition of knowledge bases for relation extraction. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1568\u20131579).","DOI":"10.3115\/v1\/D14-1165"},{"key":"9609_CR25","doi-asserted-by":"publisher","unstructured":"Guo, S., Wang, Q., Wang, L., Wang, B., & Guo, L. (2016). Jointly embedding knowledge graphs and logical rules. In J. Su, X. Carreras & K. Duh (Eds.), Proceedings of the conference on empirical methods in natural language processing (EMNLP), Austin, Texas, USA, November 1\u20134, 2016, pp. 192\u2013202. https:\/\/doi.org\/10.18653\/v1\/d16-1019","DOI":"10.18653\/v1\/d16-1019"},{"key":"9609_CR26","doi-asserted-by":"publisher","unstructured":"Hu, Z., Ma, X., Liu, Z., Hovy, E. H., & Xing, E. P. (2016). Harnessing deep neural networks with logic rules. In Proceedings of the 54th annual meeting of the association for computational linguistics, ACL 2016, August 7\u201312, 2016, Berlin, Germany, Volume 1: Long Papers. https:\/\/doi.org\/10.18653\/v1\/p16-1228","DOI":"10.18653\/v1\/p16-1228"},{"key":"9609_CR27","doi-asserted-by":"publisher","unstructured":"Hu, Z., Yang, Z., Salakhutdinov, R., & Xing, E. P. (2016). Deep neural networks with massive learned knowledge. In J. Su, X. Carreras, & K. Duh (Eds.), Proceedings of the 2016 conference on empirical methods in natural language processing, EMNLP 2016, Austin, Texas, USA, November 1\u20134, 2016, pp. 1670\u20131679. https:\/\/doi.org\/10.18653\/v1\/d16-1173","DOI":"10.18653\/v1\/d16-1173"},{"key":"9609_CR28","doi-asserted-by":"publisher","unstructured":"Nickel, M., Rosasco, L., & Poggio, T. (2016). Holographic embeddings of knowledge graphs. In Proceedings of the AAAI conference on artificial intelligence (Vol. 30). https:\/\/doi.org\/10.1609\/aaai.v30i1.10314","DOI":"10.1609\/aaai.v30i1.10314"},{"key":"9609_CR29","doi-asserted-by":"publisher","unstructured":"Diligenti, M., & Gori, M., & Sacc\u00e0, C. (2017). Semantic-based regularization for learning and inference (Vol. 244, pp. 143\u2013165). https:\/\/doi.org\/10.1016\/j.artint.2015.08.011","DOI":"10.1016\/j.artint.2015.08.011"},{"key":"9609_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1613\/jair.5714","volume":"61","author":"R Evans","year":"2018","unstructured":"Evans, R., & Grefenstette, E. (2018). Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61, 1\u201364. https:\/\/doi.org\/10.1613\/jair.5714","journal-title":"Jounal of Artificial Intelligence Research"},{"key":"9609_CR31","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1613\/jair.1.11203","volume":"62","author":"G Sourek","year":"2018","unstructured":"Sourek, G., Aschenbrenner, V., Zelezn\u00fd, F., Schockaert, S., & Kuzelka, O. (2018). Lifted relational neural networks: Efficient learning of latent relational structures. Journal of Artificial Intelligence Research, 62, 69\u2013100. https:\/\/doi.org\/10.1613\/jair.1.11203","journal-title":"Journal of Artificial Intelligence Research"},{"key":"9609_CR32","unstructured":"Sun, Z., Deng, Z.-H., Nie, J.-Y. & Tang, J. (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. CoRR arXiv:1902.10197"},{"key":"9609_CR33","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103649","volume":"303","author":"S Badreddine","year":"2022","unstructured":"Badreddine, S., d\u2019Avila Garcez, A., Serafini, L., & Spranger, M. (2022). Logic tensor networks. Artificial Intelligence, 303, 103649. https:\/\/doi.org\/10.1016\/j.artint.2021.103649","journal-title":"Artificial Intelligence"},{"key":"9609_CR34","unstructured":"Magnini, M., Ciatto, G., & Omicini, A. (2022). KINS: Knowledge injection via network structuring. In R. Calegari, G. Ciatto, & A. Omicini (Eds.), CILC 2022\u2014Italian Conference on Computational Logic CEUR Workshop Proceedings (Vol. 3204, pp. 254\u2013267). http:\/\/ceur-ws.org\/Vol-3204\/paper_25.pdf"},{"key":"9609_CR35","unstructured":"Magnini, M., Ciatto, G., & Omicini, A. (2022). A view to a KILL: Knowledge injection via lambda layer. In A. Ferrando & V. Mascardi (Eds.), WOA 2022\u201423rd workshop \u201cFrom objects to agents\u201d. CEUR workshop proceedings (Vol. 3261, pp. 61\u201376). http:\/\/ceur-ws.org\/Vol-3261\/paper5.pdf"},{"key":"9609_CR36","doi-asserted-by":"publisher","unstructured":"Marra, G., Giannini, F., Diligenti, M., & Gori, M. (2019). LYRICS: A general interface layer to integrate logic inference and deep learning. In U. Brefeld, \u00c9. Fromont, A. Hotho, A. J. Knobbe, M. H. Maathuis, & C. Robardet (Eds.), Machine learning and knowledge discovery in databases\u2014European conference, ECML PKDD 2019, W\u00fcrzburg, Germany, September 16\u201320, 2019, proceedings, Part II. Lecture Notes in Computer Science, vol. 11907, pp. 283\u2013298. Springer. https:\/\/doi.org\/10.1007\/978-3-030-46147-8_17","DOI":"10.1007\/978-3-030-46147-8_17"},{"key":"9609_CR37","unstructured":"Ruder, S. (2016). An overview of gradient descent optimization algorithms. CoRR abs\/1609.04747 arxiv:1609.04747"},{"key":"9609_CR38","doi-asserted-by":"publisher","unstructured":"Demeester, T., Rockt\u00e4schel, T. & Riedel, S. (2016). Lifted rule injection for relation embeddings. In J. Su, X. Carreras, & K. Duh (Eds.), Proceedings of the conference on empirical methods in natural language processing (EMNLP), Austin, Texas, USA, November 1\u20134, 2016, pp. 1389\u20131399 https:\/\/doi.org\/10.18653\/v1\/d16-1146","DOI":"10.18653\/v1\/d16-1146"},{"issue":"1","key":"9609_CR39","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1007\/s10994-013-5392-1","volume":"94","author":"MVM Fran\u00e7a","year":"2014","unstructured":"Fran\u00e7a, M. V. M., Zaverucha, G., & Garcez, A. S. D. (2014). Fast relational learning using bottom clause propositionalization with artificial neural networks. Machine Learning, 94(1), 81\u2013104. https:\/\/doi.org\/10.1007\/s10994-013-5392-1","journal-title":"Machine Learning"},{"key":"9609_CR40","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103504","volume":"298","author":"R Manhaeve","year":"2021","unstructured":"Manhaeve, R., Dumancic, S., Kimmig, A., Demeester, T., & De Raedt, L. (2021). Neural probabilistic logic programming in deepproblog. Artificial Intelligence, 298, 103504. https:\/\/doi.org\/10.1016\/j.artint.2021.103504","journal-title":"Artificial Intelligence"},{"key":"9609_CR41","unstructured":"Bordes, A., Usunier, N., Garc\u00eda-Dur\u00e1n, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In C. J. C. Burges, L. Bottou, Z. Ghahramani, & K. Q. Weinberger (Eds.), Proceedings of 27th annual conference on neural information processing systems (NeurIPS), Lake Tahoe, Nevada, United States, December 5\u20138, 2013, pp. 2787\u20132795. https:\/\/proceedings.neurips.cc\/paper\/2013\/hash\/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html"},{"key":"9609_CR42","unstructured":"Wang, Q., Wang, B., & Guo, L. (2015). Knowledge base completion using embeddings and rules. In Q. Yang, & M. J. Wooldridge (Eds.), Proceedings of the 24th international joint conference on artificial intelligence (IJCAI), Buenos Aires, Argentina, July 25\u201331, pp. 1859\u20131866. AAAI Press. http:\/\/ijcai.org\/Abstract\/15\/264"},{"key":"9609_CR43","unstructured":"Liu, Q., Jiang, H., Ling, Z., Wei, S., & Hu, Y. (2016) Probabilistic reasoning via deep learning: Neural association models. CoRR abs\/1603.07704 arxiv:1603.07704"},{"issue":"4","key":"9609_CR44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3447772","volume":"54","author":"A Hogan","year":"2021","unstructured":"Hogan, A., Blomqvist, E., Cochez, M., d\u2019Amato, C., Melo, G. D., Gutierrez, C., Kirrane, S., Gayo, J. E. L., Navigli, R., Neumaier, S., et al. (2021). Knowledge graphs. ACM Computing Surveys (CSUR), 54(4), 1\u201337.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"9609_CR45","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclepro.2021.125834","volume":"289","author":"T Ahmad","year":"2021","unstructured":"Ahmad, T., Zhang, D., Huang, C., Zhang, H., Dai, N., Song, Y., & Chen, H. (2021). Artificial intelligence in sustainable energy industry: Status quo, challenges and opportunities. Journal of Cleaner Production, 289, \u00a0125834. https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S0959652621000548","journal-title":"Journal of Cleaner Production"},{"key":"9609_CR46","doi-asserted-by":"publisher","unstructured":"Piedmont, R. L. (2014). In A. C. Michalos (Ed.), Bias, statistical (pp. 382\u2013383). Dordrecht: Springer. https:\/\/doi.org\/10.1007\/978-94-007-0753-5_2865","DOI":"10.1007\/978-94-007-0753-5_2865"},{"key":"9609_CR47","doi-asserted-by":"publisher","unstructured":"Liberis, E., Dudziak, L., & Lane, N. D. (2021). $$\\mu$$nas: Constrained neural architecture search for microcontrollers. In E. Yoneki, & P. Patras (Eds.), EuroMLSys@EuroSys 2021, proceedings of the 1st workshop on machine learning and Systemsg virtual event, Edinburgh, Scotland, UK, 26 April, 2021, pp. 70\u201379. ACM. https:\/\/doi.org\/10.1145\/3437984.3458836","DOI":"10.1145\/3437984.3458836"},{"key":"9609_CR48","doi-asserted-by":"publisher","unstructured":"Agiollo, A., Ciatto, G., & Omicini, A. (2021). Shallow2Deep: Restraining neural networks opacity through neural architecture search. In D. Calvaresi, A. Najjar, M. Winikoff, K. Fr\u00e4mling (Eds.), Explainable and transparent AI and multi-agent systems. Third international workshop, EXTRAAMAS 2021, virtual event, May 3\u20137, 2021, Revised Selected Papers. Lecture Notes in Computer Science (Vol. 12688, pp. 63\u201382). Cham, Switzerland: Springer. https:\/\/doi.org\/10.1007\/978-3-030-82017-6_5. http:\/\/link.springer.com\/10.1007\/978-3-030-82017-6_5","DOI":"10.1007\/978-3-030-82017-6_5"},{"key":"9609_CR49","doi-asserted-by":"publisher","unstructured":"Agiollo, A., & Omicini, A. (2021). Load classification: A case study for applying neural networks in hyper-constrained embedded devices. Applied Sciences. https:\/\/doi.org\/10.3390\/app112411957. Special Issue \u201cArtificial Intelligence and Data Engineering in Engineering Applications\u201d.","DOI":"10.3390\/app112411957"},{"key":"9609_CR50","doi-asserted-by":"publisher","unstructured":"Kang, D., Kim, E., Bae, I., Egger, B., & Ha, S. (2018). C-good: C-code generation framework for optimized on-device deep learning. In 2018 IEEE\/ACM international conference on computer-aided design (ICCAD) (pp. 1\u20138). https:\/\/doi.org\/10.1145\/3240765.3240786","DOI":"10.1145\/3240765.3240786"},{"key":"9609_CR51","doi-asserted-by":"publisher","unstructured":"Wu, B., Wan, A., Yue, X., Jin, P., Zhao, S., Golmant, N., et al. (2018) Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 9127\u20139135). https:\/\/doi.org\/10.1109\/CVPR.2018.00951","DOI":"10.1109\/CVPR.2018.00951"},{"key":"9609_CR52","doi-asserted-by":"crossref","unstructured":"Liberis, E., Dudziak, \u0141., & Lane, N. D. (2021) $$\\mu$$nas: Constrained neural architecture search for microcontrollers. In Proceedings of the 1st workshop on machine learning and systems (pp. 70\u201379).","DOI":"10.1145\/3437984.3458836"},{"key":"9609_CR53","doi-asserted-by":"publisher","unstructured":"Huang, G., Liu, S., van\u00a0der Maaten, L., & Weinberger, K. Q. (2018). Condensenet: An efficient densenet using learned group convolutions. In 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18\u201322, Computer Vision Foundation\/IEEE Computer Society, pp. 2752\u20132761. https:\/\/doi.org\/10.1109\/CVPR.2018.00291","DOI":"10.1109\/CVPR.2018.00291"},{"key":"9609_CR54","doi-asserted-by":"publisher","unstructured":"Cheng, H., Zhang, T., Yang, Y., Yan, F., Teague, H., Chen, Y., et al. (2019). Msnet: Structural wired neural architecture search for internet of things. In 2019 IEEE\/CVF international conference on computer vision workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27\u201328, pp. 2033\u20132036. IEEE. https:\/\/doi.org\/10.1109\/ICCVW.2019.00254","DOI":"10.1109\/ICCVW.2019.00254"},{"key":"9609_CR55","unstructured":"Wu, B. (2019) Efficient deep neural networks. CoRR abs\/1908.08926 arxiv:1908.08926"},{"issue":"2","key":"9609_CR56","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1109\/MCOM.001.2000200","volume":"59","author":"S Savazzi","year":"2021","unstructured":"Savazzi, S., Nicoli, M., Bennis, M., Kianoush, S., & Barbieri, L. (2021). Opportunities of federated learning in connected, cooperative, and automated industrial systems. IEEE Communications Magazine, 59(2), 16\u201321.","journal-title":"IEEE Communications Magazine"},{"key":"9609_CR57","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1016\/j.automatica.2016.09.042","volume":"75","author":"W Hou","year":"2017","unstructured":"Hou, W., Fu, M., Zhang, H., & Wu, Z. (2017). Consensus conditions for general second-order multi-agent systems with communication delay. Automatica, 75, 293\u2013298. https:\/\/doi.org\/10.1016\/j.automatica.2016.09.042","journal-title":"Automatica"},{"key":"9609_CR58","doi-asserted-by":"publisher","unstructured":"Shumailov, I., Zhao, Y., Bates, D., Papernot, N., Mullins, R. D., & Anderson, R. (2021). Sponge examples: Energy-latency attacks on neural networks. In IEEE European symposium on security and privacy, EuroS &P 2021, Vienna, Austria, September 6\u201310, pp. 212\u2013231. IEEE. https:\/\/doi.org\/10.1109\/EuroSP51992.2021.00024","DOI":"10.1109\/EuroSP51992.2021.00024"},{"key":"9609_CR59","doi-asserted-by":"publisher","unstructured":"Deng, J., & Ren, F. (2021) A survey of textual emotion recognition and its challenges. IEEE Transactions on Affective Computing, 14, 49-67. https:\/\/doi.org\/10.1109\/TAFFC.2021.3053275","DOI":"10.1109\/TAFFC.2021.3053275"},{"issue":"3","key":"9609_CR60","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/MCAS.2020.3005467","volume":"20","author":"R Sanchez-Iborra","year":"2020","unstructured":"Sanchez-Iborra, R., & Skarmeta, A. F. (2020). TinyML-enabled frugal smart objects: Challenges and opportunities. IEEE Circuits and Systems Magazine, 20(3), 4\u201318. https:\/\/doi.org\/10.1109\/MCAS.2020.3005467","journal-title":"IEEE Circuits and Systems Magazine"},{"key":"9609_CR61","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. https:\/\/www.tensorflow.org\/"},{"key":"9609_CR62","doi-asserted-by":"publisher","unstructured":"Gamma, E., Helm, R., Johnson, R. E., & Vlissides, J.M. (1993). Design patterns: Abstraction and reuse of object-oriented design. In O. Nierstrasz (Ed.), ECOOP\u201993\u2014Object-oriented programming, 7th European conference, Kaiserslautern, Germany, July 26\u201330, 1993, Proceedings. Lecture Notes in Computer Science (Vol. 707, pp. 406\u2013431). Springer. https:\/\/doi.org\/10.1007\/3-540-47910-4_21","DOI":"10.1007\/3-540-47910-4_21"},{"key":"9609_CR63","unstructured":"Wolberg, W. H. (1992). UCI machine learning repository. http:\/\/archive.ics.uci.edu\/ml"},{"issue":"1\u20132","key":"9609_CR64","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/0004-3702(94)90105-8","volume":"70","author":"GG Towell","year":"1994","unstructured":"Towell, G. G., & Shavlik, J. W. (1994). Knowledge-based artificial neural networks. Artificial Intelligence, 70(1\u20132), 119\u2013165. https:\/\/doi.org\/10.1016\/0004-3702(94)90105-8","journal-title":"Artificial Intelligence"},{"key":"9609_CR65","unstructured":"Dua, D., & Graff, C. (2017). UCI machine learning repository. http:\/\/archive.ics.uci.edu\/ml"},{"issue":"1","key":"9609_CR66","doi-asserted-by":"publisher","first-page":"27","DOI":"10.3233\/IA-210120","volume":"16","author":"F Sabbatini","year":"2022","unstructured":"Sabbatini, F., Ciatto, G., Calegari, R., & Omicini, A. (2022). Symbolic knowledge extraction from opaque ML predictors in PSyKE: Platform design & experiments. Intelligenza Artificiale, 16(1), 27\u201348. https:\/\/doi.org\/10.3233\/IA-210120","journal-title":"Intelligenza Artificiale"}],"container-title":["Autonomous Agents and Multi-Agent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-023-09609-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10458-023-09609-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-023-09609-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,17]],"date-time":"2023-12-17T23:47:34Z","timestamp":1702856854000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10458-023-09609-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,23]]},"references-count":66,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["9609"],"URL":"https:\/\/doi.org\/10.1007\/s10458-023-09609-6","relation":{},"ISSN":["1387-2532","1573-7454"],"issn-type":[{"value":"1387-2532","type":"print"},{"value":"1573-7454","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,23]]},"assertion":[{"value":"17 May 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 June 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Partial financial support was received from CHIST-ERA.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"27"}}