{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:39:48Z","timestamp":1760236788730,"version":"build-2065373602"},"reference-count":63,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,24]],"date-time":"2021-12-24T00:00:00Z","timestamp":1640304000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Modern AI technologies make use of statistical learners that lead to self-empiricist logic, which, unlike human minds, use learned non-symbolic representations. Nevertheless, it seems that it is not the right way to progress in AI. The structure of symbols\u2014the operations by which the intellectual solution is realized\u2014and the search for strategic reference points evoke important issues in the analysis of AI. Studying how knowledge can be represented through methods of theoretical generalization and empirical observation is only the latest step in a long process of evolution. For many years, humans, seeing language as innate, have carried out symbolic theories. Everything seems to have skipped ahead with the advent of Machine Learning. In this paper, after a long analysis of history, the rule-based and the learning-based vision, we would investigate the syntax as possible meeting point between the different learning theories. Finally, we propose a new vision of knowledge in AI models based on a combination of rules, learning, and human knowledge.<\/jats:p>","DOI":"10.3390\/fi14010010","type":"journal-article","created":{"date-parts":[[2021,12,27]],"date-time":"2021-12-27T01:00:54Z","timestamp":1640566854000},"page":"10","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Dis-Cover AI Minds to Preserve Human Knowledge"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8488-4146","authenticated-orcid":false,"given":"Leonardo","family":"Ranaldi","sequence":"first","affiliation":[{"name":"Department of Innovation and Information Engineering, Guglielmo Marconi University, 00193 Roma, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3288-044X","authenticated-orcid":false,"given":"Francesca","family":"Fallucchi","sequence":"additional","affiliation":[{"name":"Department of Innovation and Information Engineering, Guglielmo Marconi University, 00193 Roma, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7301-3596","authenticated-orcid":false,"given":"Fabio Massimo","family":"Zanzotto","sequence":"additional","affiliation":[{"name":"Department of Enterprise Engineering, University of Rome Tor Vergata, 00133 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Condorelli, F., Rinaudo, F., Salvadore, F., and Tagliaventi, S. (2020). A Neural Networks Approach to Detecting Lost Heritage in Historical Video. ISPRS Int. J. Geo-Inf., 9.","DOI":"10.3390\/ijgi9050297"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.patrec.2020.02.017","article-title":"Machine Learning for Cultural Heritage: A Survey","volume":"133","author":"Fiorucci","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chomsky, N. (1965). Aspects of the Theory of Syntax, The MIT Press.","DOI":"10.21236\/AD0616323"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/S0019-9958(59)90362-6","article-title":"On certain formal properties of grammars","volume":"2","author":"Chomsky","year":"1959","journal-title":"Inf. Control"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chomsky, N. (1957). Syntactic Structures, Mouton.","DOI":"10.1515\/9783112316009"},{"key":"ref_6","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2021, October 16). Attention Is All You Need, Available online: http:\/\/xxx.lanl.gov\/abs\/1706.03762."},{"key":"ref_7","unstructured":"Sinha, K., Sodhani, S., Dong, J., Pineau, J., and Hamilton, W.L. (2021, October 16). CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text, Available online: http:\/\/xxx.lanl.gov\/abs\/1908.06177."},{"key":"ref_8","unstructured":"Talmor, A., Elazar, Y., Goldberg, Y., and Berant, J. (2021, October 16). oLMpics\u2014On what Language Model Pre-training Captures, Available online: http:\/\/xxx.lanl.gov\/abs\/1912.13283."},{"key":"ref_9","unstructured":"McCoy, T., Pavlick, E., and Linzen, T. (August, January 28). Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_10","unstructured":"Lake, B.M., and Baroni, M. (2021, October 16). Generalization without Systematicity: On the Compositional Skills of Sequence-To-Sequence Recurrent Networks, Available online: http:\/\/xxx.lanl.gov\/abs\/1711.00350."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_12","unstructured":"Goldberg, Y. (2021, October 16). Assessing BERT\u2019s Syntactic Abilities, Available online: http:\/\/xxx.lanl.gov\/abs\/1901.05287."},{"key":"ref_13","unstructured":"Hewitt, J., and Manning, C.D. (2019, January 2\u20137). A Structural Probe for Finding Syntax in Word Representations. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zanzotto, F.M., Santilli, A., Ranaldi, L., Onorati, D., Tommasino, P., and Fallucchi, F. (2020, January 16\u201320). KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.","DOI":"10.18653\/v1\/2020.emnlp-main.18"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"662","DOI":"10.1002\/acp.3484","article-title":"Using network science to analyze concept maps of psychology undergraduates","volume":"33","author":"Siew","year":"2019","journal-title":"Appl. Cogn. Psychol."},{"key":"ref_16","unstructured":"Zalta, E.N. (2019). Plato\u2019s Timaeus. The Stanford Encyclopedia of Philosophy, Metaphysics Research Lab, Stanford University. [Summer 2019 ed.]."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.cognition.2004.08.004","article-title":"The faculty of language: What\u2019s special about it?","volume":"95","author":"Pinker","year":"2005","journal-title":"Cognition"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1017\/S002222670001450X","article-title":"Explaining language universals","volume":"26","author":"Newmeyer","year":"1990","journal-title":"J. Linguist."},{"key":"ref_19","unstructured":"Manning, C.D., and Sch\u00fctze, H. (1999). Foundations of Statistical Natural Language Processing, The MIT Press."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1162\/089120103322753356","article-title":"Head-Driven Statistical Models for Natural Language Parsing","volume":"29","author":"Collins","year":"2003","journal-title":"Comput. Linguist."},{"key":"ref_21","unstructured":"Settles, B., Craven, M., and Friedland, L.A. (2021, October 16). Active Learning with Real Annotation Costs. Available online: http:\/\/burrsettles.com\/pub\/settles.nips08ws.pdf."},{"key":"ref_22","unstructured":"Zalta, E.N. (2018). Aristotle\u2019s Natural Philosophy. The Stanford Encyclopedia of Philosophy, Metaphysics Research Lab, Stanford University. [Spring 2018 ed.]."},{"key":"ref_23","unstructured":"Zalta, E.N. (2020). Aristotle\u2019s Logic. The Stanford Encyclopedia of Philosophy, Metaphysics Research Lab, Stanford University. [Fall 2020 ed.]."},{"key":"ref_24","first-page":"69","article-title":"Early Language Development And Its Neural Correlates","volume":"6","author":"Bates","year":"1970","journal-title":"Handb. Neuropsychol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Elman, J.L., Bates, E.A., Johnson, M.H., Karmiloff-Smith, A., Parisi, D., and Plunkett, K. (1996). Rethinking Innateness: A Connectionist Perspective on Development, MIT Press.","DOI":"10.7551\/mitpress\/5929.001.0001"},{"key":"ref_26","unstructured":"Salton, G. (2021, October 16). Automatic text processing: The transformation, analysis, and retrieval of Reading-Addison-Wesley. Available online: http:\/\/www.iro.umontreal.ca\/~nie\/IFT6255\/Introduction.pdf."},{"key":"ref_27","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C. (2014, January 25\u201329). GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2016). Enriching Word Vectors with Subword Information. arXiv.","DOI":"10.1162\/tacl_a_00051"},{"key":"ref_30","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_31","unstructured":"Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models are Few-Shot Learners. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"153","DOI":"10.3389\/frobt.2019.00153","article-title":"Symbolic, Distributed, and Distributional Representations for Natural Language Processing in the Era of Deep Learning: A Survey","volume":"6","author":"Ferrone","year":"2020","journal-title":"Front. Robot. AI"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1017\/S0272263100009049","article-title":"Second Language Acquisition and Universal Grammar","volume":"12","author":"White","year":"1990","journal-title":"Stud. Second. Lang. Acquis."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cristianini, N., and Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Cambridge University Press.","DOI":"10.1017\/CBO9780511801389"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Moschitti, A. (2006, January 3\u20137). Making Tree Kernels practical for Natural Language Learning. Proceedings of the EACL\u201906, Trento, Italy.","DOI":"10.3115\/1614049.1614074"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Collins, M., and Duffy, N. (2002, January 7\u201312). New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073128"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Culotta, A., and Sorensen, J. (2004, January 21\u201326). Dependency Tree Kernels for Relation Extraction. Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, Barcelona, Spain.","DOI":"10.3115\/1218955.1219009"},{"key":"ref_38","unstructured":"Pighin, D., and Moschitti, A. (2010, January 15\u201316). On Reverse Feature Engineering of Syntactic Tree Kernels. Proceedings of the Fourteenth Conference on Computational Natural Language Learning, Uppsala, Sweden."},{"key":"ref_39","unstructured":"Zanzotto, F.M., and Dell\u2019Arciprete, L. (2012). Distributed Tree Kernels. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., and Salakhutdinov, R. (2019). Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. arXiv.","DOI":"10.18653\/v1\/P19-1285"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Puccetti, G., Miaschi, A., and Dell\u2019Orletta, F. (2021, January 10). How Do BERT Embeddings Organize Linguistic Knowledge?. Proceedings of the Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, Online.","DOI":"10.18653\/v1\/2021.deelio-1.6"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1590\/S0001-37652004000200002","article-title":"Innateness and the instinct to learn","volume":"76","author":"Marler","year":"2004","journal-title":"An. Acad. Bras. Ci\u00eaNcias"},{"key":"ref_43","unstructured":"Marcus, G. (2018). Innateness, AlphaZero, and Artificial Intelligence. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1111\/j.1467-7687.2007.00569.x","article-title":"Core knowledge","volume":"10","author":"Spelke","year":"2007","journal-title":"Dev. Sci."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1162\/jocn_a_00157","article-title":"Binding at Birth: The Newborn Brain Detects Identity Relations and Sequential Position in Speech","volume":"24","author":"Gervain","year":"2012","journal-title":"J. Cognitive Neurosci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1779","DOI":"10.1126\/science.1100199","article-title":"Children Creating Core Properties of Language: Evidence from an Emerging Sign Language in Nicaragua","volume":"305","author":"Senghas","year":"2004","journal-title":"Science"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1016\/j.tics.2014.07.001","article-title":"The biological basis of language: Insight from developmental grammatical impairments","volume":"18","author":"Lely","year":"2014","journal-title":"Trends Cogn. Sci."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/neco.1992.4.1.1","article-title":"Neural Networks and the Bias\/Variance Dilemma","volume":"4","author":"Geman","year":"1992","journal-title":"Neural Comput."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1162\/neco.1996.8.7.1341","article-title":"The Lack of a Priori Distinctions between Learning Algorithms","volume":"8","author":"Wolpert","year":"1996","journal-title":"Neural Comput."},{"key":"ref_50","unstructured":"Ranaldi, L., Fallucchi, F., and Zanzotto, F.M. (December, January 29). KERMITviz: Visualizing Neural Network Activations on Syntactic Trees. Proceedings of the 15th International Conference on Metadata and Semantics Research (MTSR\u201921), Madrid, Spain."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0130140","article-title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PLoS ONE"},{"key":"ref_52","unstructured":"Zanzotto, F.M., and Dell\u2019Arciprete, L. (July, January 26). Distributed tree kernels. Proceedings of the 29th International Conference on Machine Learning, ICML, Edinburgh, Scotland."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., and McClosky, D. (2014, January 23\u201324). The Stanford CoreNLP natural language processing toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD, USA.","DOI":"10.3115\/v1\/P14-5010"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2019). HuggingFace\u2019s Transformers: State-of-the-art Natural Language Processing. arxiv.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"ref_55","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled weight decay regularization. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_56","unstructured":"Agarap, A.F. (2018). Deep Learning using Rectified Linear Units (ReLU). arXiv."},{"key":"ref_57","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_58","first-page":"649","article-title":"Character-level Convolutional Networks for Text Classification","volume":"28","author":"Zhang","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_59","unstructured":"Jawahar, G., Sagot, B., and Seddah, D. (August, January 28). What Does BERT Learn about the Structure of Language?. Proceedings of the Conference of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), Florence, Italy."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., and Fidler, S. (2015, January 7\u201313). Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Washington, DC, USA.","DOI":"10.1109\/ICCV.2015.11"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Stella, M., Vitevitch, M.S., and Botta, F. (2021). Cognitive networks identify the content of English and Italian popular posts about COVID-19 vaccines: Anticipation, logistics, conspiracy and loss of trust. arXiv.","DOI":"10.3390\/bdcc6020052"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Zanzotto, F.M., Onorati, D., Tommasino, P., Ranaldi, L., and Fallucchi, F. (2020). Pat-in-the-Loop: Declarative Knowledge for Controlling Neural Networks. Future Internet, 12.","DOI":"10.3390\/fi12120218"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/1\/10\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:53:09Z","timestamp":1760169189000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/1\/10"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,24]]},"references-count":63,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["fi14010010"],"URL":"https:\/\/doi.org\/10.3390\/fi14010010","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2021,12,24]]}}}