{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T02:08:25Z","timestamp":1782439705505,"version":"3.54.5"},"reference-count":67,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,21]],"date-time":"2023-06-21T00:00:00Z","timestamp":1687305600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>People are today increasingly relying on health information they find online to make decisions that may impact both their physical and mental wellbeing. Therefore, there is a growing need for systems that can assess the truthfulness of such health information. Most of the current literature solutions use machine learning or knowledge-based approaches treating the problem as a binary classification task, discriminating between correct information and misinformation. Such solutions present several problems with regard to user decision making, among which: (i) the binary classification task provides users with just two predetermined possibilities with respect to the truthfulness of the information, which users should take for granted; indeed, (ii) the processes by which the results were obtained are often opaque and the results themselves have little or no interpretation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>To address these issues, we approach the problem as an<jats:italic>ad hoc<\/jats:italic>retrieval task rather than a classification task, with reference, in particular, to the Consumer Health Search task. To do this, a previously proposed Information Retrieval model, which considers information truthfulness as a dimension of relevance, is used to obtain a ranked list of both topically-relevant and truthful documents. The novelty of this work concerns the extension of such a model with a solution for the explainability of the results obtained, by relying on a knowledge base consisting of scientific evidence in the form of medical journal articles.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results and discussion<\/jats:title><jats:p>We evaluate the proposed solution both quantitatively, as a standard classification task, and qualitatively, through a user study to examine the \u201cexplained\u201d ranked list of documents. The results obtained illustrate the solution's effectiveness and usefulness in making the retrieved results more interpretable by Consumer Health Searchers, both with respect to topical relevance and truthfulness.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2023.1184851","type":"journal-article","created":{"date-parts":[[2023,6,21]],"date-time":"2023-06-21T10:37:45Z","timestamp":1687343865000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Explainable online health information truthfulness in Consumer Health Search"],"prefix":"10.3389","volume":"6","author":[{"given":"Rishabh","family":"Upadhyay","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Petr","family":"Knoth","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabriella","family":"Pasi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Viviani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,21]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","article-title":"Peeking inside the black-box: a survey on explainable artificial intelligence (XAI)","volume":"6","author":"Adadi","year":"2018","journal-title":"IEEE Access"},{"key":"B2","first-page":"116","article-title":"Use of the internet as a resource of health information by patients: a clinic-based study in the indian population","volume":"51","author":"Akerkar","year":"2005","journal-title":"J. Postgrad. Med."},{"key":"B3","article-title":"Explainable information retrieval: A survey","author":"Anand","year":"2022","journal-title":"arXiv preprint arXiv:2211.02405"},{"key":"B4","doi-asserted-by":"publisher","first-page":"102569","DOI":"10.1016\/j.ipm.2021.102569","article-title":"Combat COVID-19 infodemic using explainable natural language processing models","volume":"58","author":"Ayoub","year":"2021","journal-title":"Inform. Process. Manage."},{"key":"B5","doi-asserted-by":"publisher","first-page":"e0130140","DOI":"10.1371\/journal.pone.0130140","article-title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PLoS ONE"},{"key":"B6","first-page":"279","article-title":"\u201cExploring the impact of information seeking behaviors of online health consumers in the arab world,\u201d","volume-title":"Unifying the Applications and Foundations of Biomedical and Health Informatics","author":"Bahkali","year":"2016"},{"key":"B7","first-page":"1","article-title":"\u201cDoes the whole exceed its parts? The effect of AI explanations on complementary team performance,\u201d","author":"Bansal","year":"2021","journal-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems"},{"key":"B8","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1109\/THMS.2020.2965529","article-title":"Agent transparency: a review of current theory and evidence","volume":"50","author":"Bhaskara","year":"2020","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"B9","doi-asserted-by":"crossref","first-page":"1844","DOI":"10.1109\/ICMLA.2019.00297","article-title":"\u201cComprehend medical: a named entity recognition and relationship extraction web service,\u201d","volume-title":"2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA)","author":"Bhatia","year":"2019"},{"key":"B10","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1007\/s13347-019-00391-6","article-title":"Artificial intelligence and patient-centered decision-making","volume":"34","author":"Bjerring","year":"2021","journal-title":"Philos. Technol."},{"key":"B11","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1207\/s15516709cog0902_1","article-title":"An overview of the KL-ONE knowledge representation system","volume":"9","author":"Brachman","year":"1985","journal-title":"Cogn. Sci."},{"key":"B12","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/S0169-7552(98)00110-X","article-title":"The anatomy of a large-scale hypertextual web search engine","volume":"30","author":"Brin","year":"1998","journal-title":"Comput. Netw. ISDN Syst."},{"key":"B13","article-title":"Responsible AI in healthcare","author":"Cabitza","year":"2022","journal-title":"arXiv preprint arXiv:2203.03616"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1105","DOI":"10.1080\/10410236.2015.1045236","article-title":"Modeling online health information-seeking behavior in China: the roles of source characteristics, reward assessment, and internet self-efficacy","volume":"31","author":"Cao","year":"2016","journal-title":"Health Commun."},{"key":"B15","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1007\/978-3-030-69128-8_2","article-title":"\u201cTrustworthy AI,\u201d","author":"Chatila","year":"2021","journal-title":"Reflections on Artificial Intelligence for Humanity"},{"key":"B16","doi-asserted-by":"publisher","first-page":"2417","DOI":"10.1001\/jama.2018.16865","article-title":"Addressing health-related misinformation on social media","volume":"320","author":"Chou","year":"2018","journal-title":"JAMA"},{"key":"B17","article-title":"\u201cOverview of the trec 2020 health misinformation track,\u201d","author":"Clarke","year":"2020","journal-title":"TREC"},{"key":"B18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/MysuruCon55714.2022.9972435","article-title":"\u201cImproving healthcare question answering system by identifying suitable answers,\u201d","volume-title":"2022 IEEE 2nd Mysore Sub Section International Conference (MysuruCon)","author":"Das","year":"2022"},{"key":"B19","doi-asserted-by":"publisher","first-page":"3307","DOI":"10.3390\/app10093307","article-title":"A comparative analysis of machine learning methods for class imbalance in a smoking cessation intervention","volume":"10","author":"Davagdorj","year":"2020","journal-title":"Appl. Sci."},{"key":"B20","doi-asserted-by":"publisher","first-page":"2173","DOI":"10.3390\/ijerph19042173","article-title":"Health misinformation detection in the social web: an overview and a data science approach","volume":"19","author":"Di Sotto","year":"2022","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"B21","unstructured":"EU Citizens: Over Half Seek Health Information Online. Technical report, Eurostat2022"},{"key":"B22","first-page":"162","article-title":"From intermediation to disintermediation and apomediation: new models for consumers to access and assess the credibility of health information in the age of web2. 0","volume":"129","author":"Eysenbach","year":"2007","journal-title":"Stud. Health Technol. Inform."},{"key":"B23","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1145\/3299768","article-title":"The history of digital spam","volume":"62","author":"Ferrara","year":"2019","journal-title":"Commun. ACM"},{"key":"B24","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol. Bull."},{"key":"B25","unstructured":"FoxS. DugganM. Health Online 2013. Technical report, Pew Research Center2013"},{"key":"B26","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1016\/j.ijhcs.2013.12.007","article-title":"How should I explain? a comparison of different explanation types for recommender systems","volume":"72","author":"Gedikli","year":"2014","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"B27","doi-asserted-by":"publisher","first-page":"67","DOI":"10.7812\/TPP\/07-144","article-title":"Do patients understand?","volume":"12","author":"Graham","year":"2008","journal-title":"Permanente J."},{"key":"B28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3236009","article-title":"A survey of methods for explaining black box models","volume":"51","author":"Guidotti","year":"2018","journal-title":"ACM Comput. Surv."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aay7120","article-title":"XAI-explainable artificial intelligence","author":"Gunning","year":"2019","journal-title":"Sci. Robot."},{"key":"B30","doi-asserted-by":"publisher","first-page":"40","DOI":"10.12968\/bjhc.2020.0128","article-title":"Combating the spread of health misinformation on social media","volume":"27","author":"Harris","year":"2021","journal-title":"Brit. J. Healthcare Manage."},{"key":"B31","author":"Inam","year":"2021","journal-title":"Explainable AI \u2013 How Humans Can Trust AI."},{"key":"B32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s13278-020-00696-x","article-title":"Deep learning for misinformation detection on online social networks: a survey and new perspectives","volume":"10","author":"Islam","year":"2020","journal-title":"Soc. Netw. Anal. Mining"},{"key":"B33","volume-title":"Health Literacy: A Prescription to End Confusion.","author":"Kindig","year":"2004"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3492855","article-title":"HC-COVID: a hierarchical crowdsource knowledge graph approach to explainable COVID-19 misinformation detection","volume":"6","author":"Kou","year":"2022","journal-title":"Proc. ACM Hum. Comput. Interact."},{"key":"B35","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1109\/BigData50022.2020.9378019","article-title":"\u201cExfaux: a weakly-supervised approach to explainable fauxtography detection,\u201d","volume-title":"2020 IEEE International Conference on Big Data (Big Data)","author":"Kou","year":"2020"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"B37","doi-asserted-by":"publisher","first-page":"5600","DOI":"10.1109\/TII.2021.3131180","article-title":"MED-BERT: a pretraining framework for medical records named entity recognition","volume":"18","author":"Liu","year":"2021","journal-title":"IEEE Trans. Indus. Inform."},{"key":"B38","article-title":"GCAN: graph-aware co-attention networks for explainable fake news detection on social media","author":"Lu","year":"2020","journal-title":"arXiv preprint arXiv:2004.11648"},{"key":"B39","article-title":"\u201cA unified approach to interpreting model predictions,\u201d","author":"Lundberg","year":"2017","journal-title":"Advances in Neural Information Processing Systems 30"},{"key":"B40","doi-asserted-by":"publisher","first-page":"103655","DOI":"10.1016\/j.jbi.2020.103655","article-title":"The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies","volume":"113","author":"Markus","year":"2021","journal-title":"J. Biomed. Inform."},{"key":"B41","author":"Marsden","year":"2019","journal-title":"Regulating Disinformation with Artificial Intelligence: Effects of Disinformation Initiatives on Freedom of Expression and Media Pluralism"},{"key":"B42","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1145\/1282100.1282180","article-title":"\u201cFactors and effects of information credibility,\u201d","author":"McKnight","year":"2007","journal-title":"Proceedings of the Ninth International Conference on Electronic Commerce"},{"key":"B43","unstructured":"Factual"},{"key":"B44","unstructured":"Misinformation"},{"key":"B45","unstructured":"Truthful"},{"key":"B46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","article-title":"Explanation in artificial intelligence: insights from the social sciences","volume":"267","author":"Miller","year":"2019","journal-title":"Artif. Intell."},{"key":"B47","article-title":"Inexpensive domain adaptation of pretrained language models: case studies on biomedical NER and COVID-19 QA","author":"Poerner","year":"2020","journal-title":"arXiv preprint arXiv:2004.03354"},{"key":"B48","article-title":"\u201cExdocs: evidence based explainable document search,\u201d","author":"Polley","year":"2021","journal-title":"ACM SIGIR Workshop on Causality in Search and Recommendation"},{"key":"B49","doi-asserted-by":"publisher","first-page":"e20","DOI":"10.2196\/jmir.1600","article-title":"The characteristics and motivations of online health information seekers: cross-sectional survey and qualitative interview study","volume":"13","author":"Powell","year":"2011","journal-title":"J. Med. Internet Res."},{"key":"B50","first-page":"2066","article-title":"\u201cVERA: prediction techniques for reducing harmful misinformation in consumer health search,\u201d","author":"Pradeep","year":"2021","journal-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval"},{"key":"B51","article-title":"Understanding the behaviors of bert in ranking","author":"Qiao","year":"2019","journal-title":"arXiv preprint arXiv:1904.07531"},{"key":"B52","first-page":"5485","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"B53","article-title":"Explaining documents' relevance to search queries","author":"Rahimi","year":"2021","journal-title":"arXiv preprint arXiv:2111.01314"},{"key":"B54","article-title":"One word at a time: adversarial attacks on retrieval models","author":"Raval","year":"2020","journal-title":"arXiv preprint arXiv:2008.02197"},{"key":"B55","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/2939672.2939778","author":"Ribeiro","year":"2016","journal-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining"},{"key":"B56","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"B57","doi-asserted-by":"publisher","first-page":"102551","DOI":"10.1016\/j.ijhcs.2020.102551","article-title":"The effects of explainability and causability on perception, trust, and acceptance: implications for explainable AI","volume":"146","author":"Shin","year":"2021","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"B58","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1145\/3292500.3330935","article-title":"\u201cDefend: explainable fake news detection,\u201d","author":"Shu","year":"2019","journal-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"},{"key":"B59","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1145\/3289600.3290620","article-title":"\u201cEXS: explainable search using local model agnostic interpretability,\u201d","author":"Singh","year":"2019","journal-title":"Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining"},{"key":"B60","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1007\/978-3-030-85251-1_21","article-title":"\u201cOverview of the clef ehealth evaluation lab 2021,\u201d","volume-title":"Experimental IR Meets Multilinguality, Multimodality, and Interaction: 12th International Conference of the CLEF Association, CLEF 2021","author":"Suominen","year":"2021"},{"key":"B61","doi-asserted-by":"publisher","first-page":"e9","DOI":"10.2196\/jmir.5729","article-title":"Internet health information seeking and the patient-physician relationship: a systematic review","volume":"19","author":"Tan","year":"2017","journal-title":"J. Med. Internet Res."},{"key":"B62","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1016\/j.pec.2020.11.016","article-title":"The influence of online health information on health decisions: a systematic review","volume":"104","author":"Thapa","year":"2021","journal-title":"Patient Educ. Counsel."},{"key":"B63","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1071\/AH20070","article-title":"Systematic review and content analysis of australian health care substitute decision making online resources","volume":"45","author":"Tran","year":"2021","journal-title":"Austral. Health Rev."},{"key":"B64","first-page":"119","article-title":"\u201cAn unsupervised approach to genuine health information retrieval based on scientific evidence,\u201d","volume-title":"International Conference on Web Information Systems Engineering","author":"Upadhyay","year":"2022"},{"key":"B65","doi-asserted-by":"publisher","first-page":"e1209","DOI":"10.1002\/widm.1209","article-title":"Credibility in social media: opinions, news, and health information\u2013a survey","volume":"7","author":"Viviani","year":"2017","journal-title":"Wiley Interdiscipl. Rev. Data Mining Knowledge Discov."},{"key":"B66","first-page":"669","article-title":"\u201cTowards explainable search results: a listwise explanation generator,\u201d","author":"Yu","year":"2022","journal-title":"Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval"},{"key":"B67","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3395046","article-title":"A survey of fake news: fundamental theories, detection methods, and opportunities","volume":"53","author":"Zhou","year":"2020","journal-title":"ACM Comput. Surv."}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1184851\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T12:52:34Z","timestamp":1702644754000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1184851\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,21]]},"references-count":67,"alternative-id":["10.3389\/frai.2023.1184851"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1184851","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,21]]},"article-number":"1184851"}}