{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:22:08Z","timestamp":1761582128477,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T00:00:00Z","timestamp":1631577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The presented paper proposes a hybrid neural architecture that enables intelligent data analysis efficacy to be boosted in smart sensor devices, which are typically resource-constrained and application-specific. The postulated concept integrates prior knowledge with learning from examples, thus allowing sensor devices to be used for the successful execution of machine learning even when the volume of training data is highly limited, using compact underlying hardware. The proposed architecture comprises two interacting functional modules arranged in a homogeneous, multiple-layer architecture. The first module, referred to as the knowledge sub-network, implements knowledge in the Conjunctive Normal Form through a three-layer structure composed of novel types of learnable units, called L-neurons. In contrast, the second module is a fully-connected conventional three-layer, feed-forward neural network, and it is referred to as a conventional neural sub-network. We show that the proposed hybrid structure successfully combines knowledge and learning, providing high recognition performance even for very limited training datasets, while also benefiting from an abundance of data, as it occurs for purely neural structures. In addition, since the proposed L-neurons can learn (through classical backpropagation), we show that the architecture is also capable of repairing its knowledge.<\/jats:p>","DOI":"10.3390\/s21186168","type":"journal-article","created":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T21:47:21Z","timestamp":1631656041000},"page":"6168","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Boosting Intelligent Data Analysis in Smart Sensors by Integrating Knowledge and Machine Learning"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2530-0283","authenticated-orcid":false,"given":"Piotr","family":"\u0141uczak","sequence":"first","affiliation":[{"name":"Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18\/22, 90-537 \u0141\u00f3d\u017a, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6051-2962","authenticated-orcid":false,"given":"Przemys\u0142aw","family":"Kucharski","sequence":"additional","affiliation":[{"name":"Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18\/22, 90-537 \u0141\u00f3d\u017a, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8600-3760","authenticated-orcid":false,"given":"Tomasz","family":"Jaworski","sequence":"additional","affiliation":[{"name":"Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18\/22, 90-537 \u0141\u00f3d\u017a, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8761-4472","authenticated-orcid":false,"given":"Izabela","family":"Perenc","sequence":"additional","affiliation":[{"name":"Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18\/22, 90-537 \u0141\u00f3d\u017a, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1228-0970","authenticated-orcid":false,"given":"Krzysztof","family":"\u015alot","sequence":"additional","affiliation":[{"name":"Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18\/22, 90-537 \u0141\u00f3d\u017a, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8704-1950","authenticated-orcid":false,"given":"Jacek","family":"Kucharski","sequence":"additional","affiliation":[{"name":"Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18\/22, 90-537 \u0141\u00f3d\u017a, Poland"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,14]]},"reference":[{"key":"ref_1","unstructured":"Hernandez, D., and Brown, T.B. (2020). Measuring the Algorithmic Efficiency of Neural Networks. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go through Self-Play","volume":"6419","author":"Silver","year":"2018","journal-title":"Science"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1038\/s41586-019-1923-7","article-title":"Improved Protein Structure Prediction Using Potentials from Deep Learning","volume":"577","author":"Senior","year":"2020","journal-title":"Nature"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly Accurate Protein Structure Prediction with AlphaFold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"ref_5","first-page":"9","article-title":"Language Models Are Unsupervised Multitask Learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"ref_6","unstructured":"Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models Are Few-Shot Learners. arXiv."},{"key":"ref_7","unstructured":"Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Man\u00e9, D. (2016). Concrete Problems in AI Safety. arXiv."},{"key":"ref_8","unstructured":"Lillicrap, T.P., and Kording, K.P. (2019). What Does It Mean to Understand a Neural Network?. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI","volume":"58","author":"Arrieta","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-020-0262-2","article-title":"The Need for a System View to Regulate Artificial Intelligence\/Machine Learning-Based Software as Medical Device","volume":"3","author":"Gerke","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1109\/MWC.001.1900534","article-title":"Holographic MIMO Surfaces for 6G Wireless Networks: Opportunities, Challenges, and Trends","volume":"27","author":"Huang","year":"2020","journal-title":"IEEE Wirel. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/BF02551274","article-title":"Approximation by Superpositions of a Sigmoidal Function","volume":"2","author":"Cybenko","year":"1989","journal-title":"Math. Control. Signals Syst."},{"key":"ref_13","first-page":"2","article-title":"On the Representation of Continuous Functions of Several Variables by Superpositions of Continuous Functions of Lesser Variable Count","volume":"108","author":"Kolmogorov","year":"1956","journal-title":"Dokl. Akad. Nauk SSSR"},{"key":"ref_14","first-page":"679","article-title":"On Functions of Three Variables","volume":"Volume 114","author":"Arnold","year":"1957","journal-title":"Doklady Akademii Nauk"},{"key":"ref_15","unstructured":"Gori, M. (2017). Machine Learning: A Constraint-Based Approach, Morgan Kaufmann."},{"key":"ref_16","unstructured":"Roychowdhury, S., Diligenti, M., and Gori, M. (2018, January 2\u20137). Image Classification Using Deep Learning and Prior Knowledge. Proceedings of the Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/0004-3702(94)90105-8","article-title":"Knowledge-Based Artificial Neural Networks","volume":"70","author":"Towell","year":"1994","journal-title":"Artif. Intell."},{"key":"ref_18","unstructured":"Gaier, A., and Ha, D. (2019, January 8\u201314). Weight Agnostic Neural Networks. Proceedings of the NeurIPS 2019: Thirty-Third Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1080\/09540098908915640","article-title":"An Approach to Combining Explanation-Based and Neural Learning Algorithms","volume":"1","author":"Shavlik","year":"1989","journal-title":"Connect. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/BF00993103","article-title":"Extracting Refined Rules from Knowledge-Based Neural Networks","volume":"13","author":"Towell","year":"1993","journal-title":"Mach. Learn."},{"key":"ref_21","unstructured":"Riegel, R., Gray, A., Luus, F., Khan, N., Makondo, N., Akhalwaya, I.Y., Qian, H., Fagin, R., Barahona, F., and Sharma, U. (2020). Logical Neural Networks. arXiv."},{"key":"ref_22","unstructured":"Marra, G., Diligenti, M., Giannini, F., Gori, M., and Maggini, M. (2020). Relational Neural Machines. arXiv."},{"key":"ref_23","first-page":"1500","article-title":"Deep Reasoning Networks for Unsupervised Pattern De-Mixing with Constraint Reasoning","volume":"119","author":"Chen","year":"2020","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_24","unstructured":"Herrmann, C., and Thier, A. (1996). Backpropagation for Neural DNF- and CNF-Networks. Knowl. Represent. Neural Networks, 63\u201372."},{"key":"ref_25","first-page":"5338","article-title":"Concept Bottleneck Models","volume":"119","author":"Koh","year":"2020","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1162\/106365602320169811","article-title":"Evolving Neural Networks through Augmenting Topologies","volume":"10","author":"Stanley","year":"2002","journal-title":"Evol. Comput."},{"key":"ref_27","unstructured":"Wang, P.W., Donti, P.L., Wilder, B., and Kolter, Z. (2019). SATNet: Bridging Deep Learning and Logical Reasoning Using a Differentiable Satisfiability Solver. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1109\/21.256541","article-title":"ANFIS: Adaptive-Network-Based Fuzzy Inference System","volume":"23","author":"Jang","year":"1993","journal-title":"IEEE Trans. Syst. Man. Cybern."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/BF02478259","article-title":"A Logical Calculus of the Ideas Immanent in Nervous Activity","volume":"5","author":"McCulloch","year":"1943","journal-title":"Bull. Math. Biophys."},{"key":"ref_30","unstructured":"Kingma, D.P., and Ba, J.L. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Czogala, E., and Leski, J. (2000). Fuzzy and Neuro-Fuzzy Intelligent Systems, Physica-Verlag HD.","DOI":"10.1007\/978-3-7908-1853-6"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/0165-0114(95)00133-6","article-title":"Uninorm Aggregation Operators","volume":"80","author":"Yager","year":"1996","journal-title":"Fuzzy Sets Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MCOM.001.2001146","article-title":"AI-Assisted MAC for Reconfigurable Intelligent-Surface-Aided Wireless Networks: Challenges and Opportunities","volume":"59","author":"Cao","year":"2021","journal-title":"IEEE Commun. Mag."},{"key":"ref_34","unstructured":"Ng, A.Y. (2004). Feature Selection, L1 vs. L2 Regularization, and Rotational Invariance. Proceedings of the Twenty-First International Conference on Machine Learning, Association for Computing Machinery."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MIS.2009.36","article-title":"The Unreasonable Effectiveness of Data","volume":"24","author":"Halevy","year":"2009","journal-title":"IEEE Intell. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Banko, M., and Brill, E. (2001, January 6\u201311). Scaling to Very Very Large Corpora for Natural Language Disambiguation. Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, ACL \u201901, Toulouse, France.","DOI":"10.3115\/1073012.1073017"},{"key":"ref_37","unstructured":"Mosley, L. (2013). A Balanced Approach to the Multi-Class Imbalance Problem. [Ph.D. Thesis, Iowa State University]."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3","article-title":"Index for Rating Diagnostic Tests","volume":"3","author":"Youden","year":"1950","journal-title":"Cancer"},{"key":"ref_39","unstructured":"Kahneman, D. (2011). Thinking, Fast and Slow, Farrar, Straus and Giroux."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6168\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:02:42Z","timestamp":1760166162000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6168"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,14]]},"references-count":39,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21186168"],"URL":"https:\/\/doi.org\/10.3390\/s21186168","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,9,14]]}}}