{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:14:17Z","timestamp":1760148857298,"version":"build-2065373602"},"reference-count":69,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T00:00:00Z","timestamp":1686528000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In spite of great progress in recent years, deep learning (DNN) and transformers have strong limitations for supporting human\u2013machine teams due to a lack of explainability, information on what exactly was generalized, and machinery to be integrated with various reasoning techniques, and weak defense against possible adversarial attacks of opponent team members. Due to these shortcomings, stand-alone DNNs have limited support for human\u2013machine teams. We propose a Meta-learning\/DNN \u2192 kNN architecture that overcomes these limitations by integrating deep learning with explainable nearest neighbor learning (kNN) to form the object level, having a deductive reasoning-based meta-level control learning process, and performing validation and correction of predictions in a way that is more interpretable by peer team members. We address our proposal from structural and maximum entropy production perspectives.<\/jats:p>","DOI":"10.3390\/e25060924","type":"journal-article","created":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T02:56:34Z","timestamp":1686624994000},"page":"924","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Shaped-Charge Learning Architecture for the Human\u2013Machine Teams"],"prefix":"10.3390","volume":"25","author":[{"given":"Boris","family":"Galitsky","sequence":"first","affiliation":[{"name":"Knowledge-Trail, San Jose, CA 93635, USA"}]},{"given":"Dmitry","family":"Ilvovsky","sequence":"additional","affiliation":[{"name":"Computer Science Faculty, HSE University, Moscow 101000, Russia"}]},{"given":"Saveli","family":"Goldberg","sequence":"additional","affiliation":[{"name":"Department of Radiology at Massachusetts General Hospital, Boston, MA 02114, USA"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"30","DOI":"10.3389\/fphy.2017.00030","article-title":"The Physics of Teams: Interdependence, Measurable Entropy, and Computational Emotion","volume":"5","author":"Lawless","year":"2017","journal-title":"Front. Phys."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Galitsky, B., and Goldberg, S. (2019). Developing Enterprise Chatbots, Springer.","DOI":"10.1007\/978-3-030-04299-8"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Galitsky, B. (2019). Assuring Chatbot Relevance at Syntactic Level, in Developing Enterprise Chatbots, Springer.","DOI":"10.1007\/978-3-030-04299-8"},{"key":"ref_4","unstructured":"Lawless, W., Mittu, R., and Sofge, D. (2020). Human-Machine Shared Contexts, Elsevier."},{"key":"ref_5","first-page":"5","article-title":"Artificial intelligence, Autonomy, and Human-Machine Teams\u2014Interdependence, Context, and Explainable AI","volume":"40","author":"Lawless","year":"2019","journal-title":"AI Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s42454-021-00031-5","article-title":"A bi-directional adversarial explainability for decision support","volume":"3","author":"Goldberg","year":"2021","journal-title":"Hum. Intell. Syst. Integr."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1016\/j.tics.2007.09.004","article-title":"Learning multiple layers of representation","volume":"11","author":"Hinton","year":"2007","journal-title":"Trends Cogn. Sci."},{"key":"ref_8","unstructured":"Lawless, W.F., Mittu, R., and Sofge, D.A. (2020). Human-Machine Shared Contexts, Academic Press."},{"key":"ref_9","unstructured":"Bilton, N. (2022, December 01). She Absolutely Has Sociopathic. VanityFair. Tendencies. Available online: https:\/\/www.vanityfair.com\/news\/2018\/06\/elizabeth-holmes-is-trying-to-start-a-new-company."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Galitsky, B. (2020). Artificial Intelligence for Customer Relationship Management, Springer.","DOI":"10.1007\/978-3-030-52167-7"},{"key":"ref_11","unstructured":"SEC (2022, December 01). US Securities and Exchange Commission. Theranos, CEO Holmes, and Former President Balwani Charged with Massive Fraud, Available online: https:\/\/www.sec.gov\/news\/press-release\/2018-41."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shrestha, A., and Mahmood, A. (2019). Review of Deep Learning Algorithms and Architectures, IEEE.","DOI":"10.1109\/ACCESS.2019.2912200"},{"key":"ref_13","unstructured":"Chollet, F. (2022, December 01). The Limitations of Deep Learning. The Keras Blog. Available online: https:\/\/blog.keras.io\/the-limitations-of-deep-learning.html."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1007\/s13748-021-00239-1","article-title":"Deep limitations? Examining expert disagreement over deep learning","volume":"10","author":"Cremer","year":"2021","journal-title":"Prog. Artif. Intell."},{"key":"ref_15","unstructured":"Galitsky, B. (2015). FCA4AI@ IJCAI, HAL (Open Archive)."},{"key":"ref_16","unstructured":"Papernot, N., and Mcdaniel, P. (2018). Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lin, S., Hilton, J., and Evans, O. (2021). TruthfulQA: Measuring how models mimic human falsehoods. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.229"},{"key":"ref_18","unstructured":"Petroni, F., Lewis, P., Piktus, A., Rockta, T., Wu, Y., Miller, H., and Riedel, S. (2022, December 01). How Context Affects Language Models\u2019 Factual Predictions. In Automated Knowledge Base Construction. Available online: https:\/\/openreview.net\/forum?id=025X0zPfn."},{"key":"ref_19","unstructured":"Galitsky, B., and Golberg, C. (2022). Artificial Intelligence for Healthcare Applications and Management, Elsevier."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e253","DOI":"10.1017\/S0140525X16001837","article-title":"Building machines that earn and think like people","volume":"40","author":"Lake","year":"2016","journal-title":"Behav. Brain Sci."},{"key":"ref_21","first-page":"3649","article-title":"Efficient program synthesis using constraint satisfaction in inductive logic programming","volume":"14","author":"Ahlgren","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cropper, A., Duman\u010di\u0107, S., and Muggleton, S.H. (2020, January 11\u201317). Turning30: New ideas in inductive logic programming. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, Yokohama, Japan.","DOI":"10.24963\/ijcai.2020\/673"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1017\/S1471068418000261","article-title":"Exploiting answer set programming with external sources for meta-interpretive learning","volume":"18","author":"Kaminski","year":"2018","journal-title":"Theory Pract. Log. Program."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"De Raedt, L., Dumancic, S., Manhaeve, R., and Marra, G. (2020, January 11\u201317). From Statistical Relational to Neuro-Symbolic Artificial Intelligence. Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020), Yokohama, Japan.","DOI":"10.24963\/ijcai.2020\/688"},{"key":"ref_25","unstructured":"d\u2019Garcez, A.S., Lamban, L.C., and Gabbay, D.M. (2009). Neural-Symbolic Cognitive Reasoning, Cognitive Technologies, Springer."},{"key":"ref_26","unstructured":"Muggleton, S. (1992). Inductive Logic Programming, Academic Press."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Nienhuys-Cheng, S.-H., Wolf, R.d., Siekmann, J., and Carbonell, J.G. (1997). Foundations of Inductive Logic Programming, Springer.","DOI":"10.1007\/3-540-62927-0"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1007\/s10994-020-05934-z","article-title":"Learning programs by learning from failures","volume":"110","author":"Cropper","year":"2021","journal-title":"Mach. Learn."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, W.Y., Mazaitis, K., and Cohen, W. (2014, January 3\u20137). Structure learning via parameter learning. Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM 2014, Shanghai, China.","DOI":"10.1145\/2661829.2662022"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Evans, R., and Grefenstette, E. (2018). Learning Explanatory Rules from Noisy Data. IJCAI, 5598\u20135602.","DOI":"10.24963\/ijcai.2018\/792"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1007\/s10994-019-05834-x","article-title":"Logical reduction of metarules","volume":"109","author":"Cropper","year":"2020","journal-title":"Mach. Learn."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Shindo, H., Nishino, M., and Yamamoto, A. (2021, January 2\u20139). Differentiable Inductive Logic Programming for Structured Examples. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v35i6.16637"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Fix, E., and Hodges, J.L. (1951). Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties (PDF) (Report), USAF School of Aviation Medicine.","DOI":"10.1037\/e471672008-001"},{"key":"ref_34","unstructured":"Sabour, S., Cao, Y., Faghri, F., and Fleet, D.J. (2016). Adversarial Manipulation of Deep Representations, Cornell University."},{"key":"ref_35","unstructured":"Engstrom, L., Tsipras, D., Schmidt, L., and Madry, A. (2017). A rotation and a translation suffice: Fooling cnns with simple transformations. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy should I trust you?\u201d: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by back- propagating errors","volume":"323","author":"Rumelhart","year":"1986","journal-title":"Nature"},{"key":"ref_38","unstructured":"Domingos, P. (2020). Every Model Learned by Gradient Descent Is Approximately a Kernel Machine. arXiv."},{"key":"ref_39","unstructured":"Lawless, W.F., Mittu, R., and Sofge, D.A. (2020). Human-Machine Shared Contexts, Academic Press."},{"key":"ref_40","unstructured":"Batali, J. (1983). Computational Introspection MIT, AI Memo."},{"key":"ref_41","unstructured":"Maes, P., and Nardi, D. (1988). Meta-Level Architectures and Reflections, North- Holland."},{"key":"ref_42","unstructured":"Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., and Schneider, N. (2013, January 8\u20139). Abstract meaning representation for sembanking. Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, Sofia, Bulgaria."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.cobeha.2019.01.005","article-title":"Doing more with less: Meta-reasoning and meta-learning in humans and machines","volume":"29","author":"Griffiths","year":"2019","journal-title":"Curr. Opin. Behav. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1037\/0033-295X.101.4.608","article-title":"A Rational Analysis of the Selection Task as Optimal Data Selection","volume":"101","author":"Oaksford","year":"1994","journal-title":"Psychol. Rev."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0004-3702(97)00026-X","article-title":"Rationality and intelligence","volume":"94","author":"Russell","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1016\/j.tics.2017.05.004","article-title":"Meta-Reasoning: Monitoring and Control of Thinking and Reasoning","volume":"21","author":"Ackerman","year":"2017","journal-title":"Trends Cogn. Sci."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Evans, J., and Frankish, K. (2009). Two Minds: Dual Processes and Beyond, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199230167.001.0001"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1080\/13546783.2013.869763","article-title":"Conflict, metacognition, and analytic thinking","volume":"20","author":"Thompson","year":"2014","journal-title":"Think. Reason."},{"key":"ref_49","unstructured":"(2022, December 01). Apache Airflow. Available online: https:\/\/airflow.apache.org."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Galitsky, B. (2021). Artificial Intelligence for Customer Relationship Management, Springer.","DOI":"10.1007\/978-3-030-61641-0"},{"key":"ref_51","unstructured":"Mohiuddin, T., Joty, S., and Nguyen, D.T. (2018). Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach, Cornell University."},{"key":"ref_52","unstructured":"Nguyen, T.D., and Joty, S. (2017). Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics."},{"key":"ref_53","unstructured":"Qubole (2022, December 01). The Ultimate Guide to Apache Airflow. Available online: https:\/\/www.qubole.com\/the-ultimate-guide-to-apache-airflow."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., and Song, D. (2020, January 13). Pretrained Transformers Improve Out-of-Distribution Robustness. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.244"},{"key":"ref_55","first-page":"821","article-title":"Theoretical foundations of the potential function method in pattern recognition learning","volume":"25","author":"Aizerman","year":"1964","journal-title":"Autom. Remote Contr."},{"key":"ref_56","unstructured":"Lippmann, R., Gold, B., and Malpass, M. (1987). Technical Report 769, MIT Lincoln Lab."},{"key":"ref_57","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Meta Fundamental AI Research Diplomacy Team (FAIR), Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., and Hu, H. (2022). Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science, 378, 1067\u20131074.","DOI":"10.1126\/science.ade9097"},{"key":"ref_59","unstructured":"Schmidhuber, J. (2022). Annotated History of Modern AI and Deep Learning. arXiv."},{"key":"ref_60","unstructured":"Clune, J. (2020). AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence. arXiv."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Dudyrev, E., Semenkov, I., Kuznetsov, S.O., Gusev, G., Sharp, A., and Pianykh, O.S. (2022). Human knowledge models: Learning applied knowledge from the data. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0275814"},{"key":"ref_62","unstructured":"Marcus, G. (2023). What to Expect When You\u2019re Expecting, Workman Publishing Company."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1111\/j.1467-7687.2007.00585.x","article-title":"Learning overhypotheses with hierarchical Bayesian models","volume":"10","author":"Kemp","year":"2007","journal-title":"Dev. Sci."},{"key":"ref_64","unstructured":"Tamkin, A., Nguyen, D., Deshpande, S., and Goodman, N. (2022, December 01). Active Learning Helps Pretrained Models Learn the Intended Task. Available online: https:\/\/arxiv.org\/abs\/2204.08491."},{"key":"ref_65","unstructured":"Mill, J.S. (2022, December 01). System of Logic. Available online: https:\/\/en.wikipedia.org\/wiki\/A_System_of_Logic."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Kovaleva, O., Kulshreshtha, S., Rogers, A., and Rumshisky, A. (2021). BERT Busters: Outlier Dimensions that Disrupt Transformers, Cornell University.","DOI":"10.18653\/v1\/2021.findings-acl.300"},{"key":"ref_67","unstructured":"Vasishth, S., Velickovski, S., Stoyanchev, M., and Frank, S. (2020, January 8\u201313). Training GPT-2 for SlavicLanguage Tasks. Proceedings of the 27th International Conference on Computational Linguistics (COLING 2020), Barcelona, Spain."},{"key":"ref_68","unstructured":"Gast, A., Anderson, N., and Technical, R. (2019). Language Models are Few-Shot Learners. arXiv."},{"key":"ref_69","unstructured":"Huck, M., Mandt, S., and S\u00f8nderby, C.K. (2020). Language Models are Zero-Shot Learners. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/6\/924\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:53:13Z","timestamp":1760125993000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/6\/924"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,12]]},"references-count":69,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["e25060924"],"URL":"https:\/\/doi.org\/10.3390\/e25060924","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,6,12]]}}}