{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T22:24:18Z","timestamp":1779315858626,"version":"3.51.4"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,10,6]],"date-time":"2022-10-06T00:00:00Z","timestamp":1665014400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,6]],"date-time":"2022-10-06T00:00:00Z","timestamp":1665014400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007065","name":"Universit\u00e0 degli Studi di Salerno","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007065","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2023,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The spreading of machine learning (ML) and deep learning (DL) methods in different and critical application domains, like medicine and healthcare, introduces many opportunities but raises risks and opens ethical issues, mainly attaining to the lack of transparency. This contribution deals with the lack of transparency of ML and DL models focusing on the lack of trust in predictions and decisions generated. In this sense, this paper establishes a measure, namely <jats:italic>Congruity<\/jats:italic>, to provide information about the reliability of ML\/DL model results. <jats:italic>Congruity<\/jats:italic> is defined by the lattice extracted through the formal concept analysis built on the training data. It measures how much the incoming data items are close to the ones used at the training stage of the ML and DL models. The general idea is that the reliability of trained model results is highly correlated with the similarity of input data and the training set. The objective of the paper is to demonstrate the correlation between the <jats:italic>Congruity<\/jats:italic> and the well-known <jats:italic>Accuracy<\/jats:italic> of the whole ML\/DL model. Experimental results reveal that the value of correlation between <jats:italic>Congruity<\/jats:italic> and <jats:italic>Accuracy<\/jats:italic> of ML model is greater than 80% by varying ML models.<\/jats:p>","DOI":"10.1007\/s00521-022-07853-7","type":"journal-article","created":{"date-parts":[[2022,10,6]],"date-time":"2022-10-06T15:10:23Z","timestamp":1665069023000},"page":"1899-1913","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Toward reliable machine learning with Congruity: a quality measure based on formal concept analysis"],"prefix":"10.1007","volume":"35","author":[{"given":"Carmen","family":"De Maio","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giuseppe","family":"Fenza","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mariacristina","family":"Gallo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4807-8942","authenticated-orcid":false,"given":"Vincenzo","family":"Loia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claudio","family":"Stanzione","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,10,6]]},"reference":[{"key":"7853_CR1","unstructured":"Commission E (2020) White paper on artificial intelligence-a European approach to excellence and trust. Com 65 Final (2020)"},{"key":"7853_CR2","doi-asserted-by":"crossref","unstructured":"Andrade NNGd, Kontschieder V (2021) Ai impact assessment: A policy prototyping experiment. Available at SSRN 3772500","DOI":"10.2139\/ssrn.3772500"},{"key":"7853_CR3","doi-asserted-by":"crossref","unstructured":"Goebel R, Chander A, Holzinger K, Lecue F, Akata Z, Stumpf S, Kieseberg P, Holzinger A (2018) Explainable AI: the new 42? In: International cross-domain conference for machine learning and knowledge extraction, pp 295\u2013303. Springer","DOI":"10.1007\/978-3-319-99740-7_21"},{"issue":"2","key":"7853_CR4","first-page":"44","volume":"40","author":"D Gunning","year":"2019","unstructured":"Gunning D, Aha D (2019) Darpa\u2019s explainable artificial intelligence (XAI) program. AI Mag 40(2):44\u201358","journal-title":"AI Mag"},{"key":"7853_CR5","unstructured":"Doran D, Schulz S, Besold TR (2017) What does explainable AI really mean? A new conceptualization of perspectives. arXiv preprint arXiv:1710.00794"},{"key":"7853_CR6","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"AB Arrieta","year":"2020","unstructured":"Arrieta AB, D\u00edaz-Rodr\u00edguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garc\u00eda S, Gil-L\u00f3pez S, Molina D, Benjamins R et al (2020) Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion 58:82\u2013115","journal-title":"Inf Fusion"},{"key":"7853_CR7","volume-title":"Four principles of explainable artificial intelligence","author":"PJ Phillips","year":"2020","unstructured":"Phillips PJ, Hahn CA, Fontana PC, Broniatowski DA, Przybocki MA (2020) Four principles of explainable artificial intelligence. Gaithersburg, Maryland"},{"key":"7853_CR8","unstructured":"DeVries T, Taylor GW (2018) Learning confidence for out-of-distribution detection in neural networks. arXiv preprint arXiv:1802.04865"},{"key":"7853_CR9","doi-asserted-by":"crossref","unstructured":"Ganter B, Stumme G, Wille R (2005) Formal concept analysis: foundations and applications 3626","DOI":"10.1007\/978-3-540-31881-1"},{"key":"7853_CR10","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) Why should i trust you?: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135\u20131144","DOI":"10.1145\/2939672.2939778"},{"key":"7853_CR11","unstructured":"Shrikumar A, Greenside P, Shcherbina A, Kundaje A (2016) Not just a black box: Learning important features through propagating activation differences. arXiv Preprint arXiv:1605.01713"},{"issue":"3","key":"7853_CR12","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/s10115-013-0679-x","volume":"41","author":"E \u0160trumbelj","year":"2014","unstructured":"\u0160trumbelj E, Kononenko I (2014) Explaining prediction models and individual predictions with feature contributions. Knowl Inf Syst 41(3):647\u2013665","journal-title":"Knowl Inf Syst"},{"key":"7853_CR13","doi-asserted-by":"crossref","unstructured":"Datta A, Sen S, Zick Y (2016) Algorithmic transparency via quantitative input influence: theory and experiments with learning systems. In: 2016 IEEE symposium on security and privacy (SP). IEEE, pp 598\u2013617","DOI":"10.1109\/SP.2016.42"},{"key":"7853_CR14","unstructured":"Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. In: NIPS\u201917: proceedings of the 31st international conference on neural information processing systems, pp 4768\u20134777"},{"key":"7853_CR15","doi-asserted-by":"crossref","unstructured":"Caruana R, Lou Y, Gehrke J, Koch P, Sturm M, Elhadad N (2015) Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In: Proceedings of the 21th ACM SIGKDD International conference on knowledge discovery and data mining, ser. KDD \u201915 New York, NY, USA: ACM, pp 1721\u20131730","DOI":"10.1145\/2783258.2788613"},{"key":"7853_CR16","unstructured":"Letham B, Rudin C, McCormick TH, Madigan D (2013) An interpretable stroke prediction model using rules and Bayesian analysis. In: Proceedings of the 17th AAAI conference on late-breaking developments in the field of artificial intelligence, ser. AAAIWS\u201913-17. AAAI Press, pp 65\u201367"},{"key":"7853_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2021.107366","volume":"106","author":"G Fenza","year":"2021","unstructured":"Fenza G, Gallo M, Loia V, Orciuoli F, Herrera-Viedma E (2021) Data set quality in machine learning: consistency measure based on group decision making. Appl Soft Comput 106:107366","journal-title":"Appl Soft Comput"},{"key":"7853_CR18","unstructured":"Schulam P, Saria S (2019) Can you trust this prediction? auditing pointwise reliability after learning. In: The 22nd international conference on artificial intelligence and statistics, pp 1022\u20131031. PMLR"},{"key":"7853_CR19","doi-asserted-by":"crossref","unstructured":"Nicora G, Rios M, Abu-Hanna A, Bellazzi R (2022) Evaluating pointwise reliability of machine learning prediction. J Biomed Inf, 103996","DOI":"10.1016\/j.jbi.2022.103996"},{"key":"7853_CR20","first-page":"1803","volume":"11","author":"D Baehrens","year":"2010","unstructured":"Baehrens D, Schroeter T, Harmeling S, Kawanabe M, Hansen K, Muller KR (2010) How to explain individual classification decisions. J Mach Learn Res 11:1803\u20131831","journal-title":"J Mach Learn Res"},{"key":"7853_CR21","unstructured":"Craven MW, Shavlik JW (1995) Extracting tree-structured representations of trained networks. In: Proceedings of the 8th international conference on neural information processing systems, ser. NIPS\u201995. Cambridge, MA, USA: MIT Press, pp 24\u201330"},{"key":"7853_CR22","doi-asserted-by":"crossref","unstructured":"et al., S.B (2015) On pixel-wise explanations for non-linear classifier decisions by layerwise relevance propagation. PloS ONE 10.7","DOI":"10.1371\/journal.pone.0130140"},{"key":"7853_CR23","unstructured":"Sangroya A, Anantaram C, Rawat M, Rastogi M (2019) Using formal concept analysis to explain black box deep learning classification models. In: FCA4AI@IJCAI 2019"},{"key":"7853_CR24","unstructured":"Hendrycks D, Gimpel K (2016) A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136"},{"key":"7853_CR25","doi-asserted-by":"publisher","unstructured":"de Moraes RF, Evangelista RdS, Fernandes LAF, Mart\u00ed L (2021) generic coupled out-of-distribution detector for robust classification. In: 2021 34th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), pp 409\u2013416. https:\/\/doi.org\/10.1109\/SIBGRAPI54419.2021.00062","DOI":"10.1109\/SIBGRAPI54419.2021.00062"},{"key":"7853_CR26","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1109\/LSP.2020.2988140","volume":"27","author":"C Yu","year":"2020","unstructured":"Yu C, Zhu X, Lei Z, Li SZ (2020) Out-of-distribution detection for reliable face recognition. IEEE Signal Process Lett 27:710\u2013714. https:\/\/doi.org\/10.1109\/LSP.2020.2988140","journal-title":"IEEE Signal Process Lett"},{"key":"7853_CR27","doi-asserted-by":"publisher","first-page":"1258","DOI":"10.1016\/j.proeng.2014.03.117","volume":"69","author":"F \u0160kopljanac-Ma\u010dina","year":"2014","unstructured":"\u0160kopljanac-Ma\u010dina F, Bla\u0161kovi\u0107 B (2014) Formal concept analysis-overview and applications. Procedia Eng 69:1258\u20131267","journal-title":"Procedia Eng"},{"issue":"24","key":"7853_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.17485\/ijst\/2015\/v8i31\/84085","volume":"8","author":"M Priya","year":"2015","unstructured":"Priya M, Kumar CA (2015) A survey of state of the art of ontology construction and merging using formal concept analysis. Indian J Sci Technol 8(24):1\u20137","journal-title":"Indian J Sci Technol"},{"issue":"9","key":"7853_CR29","doi-asserted-by":"publisher","first-page":"228","DOI":"10.3390\/info9090228","volume":"9","author":"Z Zhang","year":"2018","unstructured":"Zhang Z, Zhao J, Yan X (2018) A web page clustering method based on formal concept analysis. Information 9(9):228","journal-title":"Information"},{"key":"7853_CR30","doi-asserted-by":"crossref","unstructured":"Ferr\u00e9 S, Huchard M, Kaytoue M, Kuznetsov SO, Napoli A (2020) Formal concept analysis: from knowledge discovery to knowledge processing, pp 411\u2013445","DOI":"10.1007\/978-3-030-06167-8_13"},{"key":"7853_CR31","unstructured":"Maio CD, Fenza G, Loia V, Parente M (2015) Biomedical data integration and ontology-driven multi-facets visualization. IJCNN, pp 1\u20138"},{"issue":"5","key":"7853_CR32","doi-asserted-by":"publisher","first-page":"1763","DOI":"10.1213\/ANE.0000000000002864","volume":"126","author":"P Schober","year":"2018","unstructured":"Schober P, Boer C, Schwarte LA (2018) Correlation coefficients: appropriate use and interpretation. Anesth Analg 126(5):1763\u20131768","journal-title":"Anesth Analg"},{"key":"7853_CR33","unstructured":"Khosravi P, Vergari A, Choi Y, Liang Y, Broeck GVd (2020) Handling missing data in decision trees: a probabilistic approach. arXiv preprint arXiv:2006.16341"},{"key":"7853_CR34","doi-asserted-by":"crossref","unstructured":"Hirota K, Bede B, Nobuhara H (2006) A hierarchical representation of video\/image database by formal concept analysis and fuzzy clustering. In: 2006 IEEE international conference on fuzzy systems, pp 902\u2013906 . IEEE","DOI":"10.1109\/FUZZY.2006.1681818"},{"key":"7853_CR35","doi-asserted-by":"publisher","first-page":"86","DOI":"10.29007\/rp6q","volume":"83","author":"M Khatri","year":"2022","unstructured":"Khatri M, Voshall A, Batra SK, Kaur S, Deogun JS (2022) Interpretable image classification model using formal concept analysis based classifier. EPiC Ser Comput 83:86\u201395","journal-title":"EPiC Ser Comput"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-022-07853-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-022-07853-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-022-07853-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T17:30:53Z","timestamp":1673285453000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-022-07853-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,6]]},"references-count":35,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["7853"],"URL":"https:\/\/doi.org\/10.1007\/s00521-022-07853-7","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,6]]},"assertion":[{"value":"31 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author states that this article has no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}