{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T08:59:37Z","timestamp":1785142777208,"version":"3.55.0"},"reference-count":39,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T00:00:00Z","timestamp":1784851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T00:00:00Z","timestamp":1784851200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100000156","name":"Fonds de Recherche du Qu\u00e9bec - Sant\u00e9","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000156","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000024","name":"Canadian Institutes of Health Research","doi-asserted-by":"publisher","award":["PJT-438778"],"award-info":[{"award-number":["PJT-438778"]}],"id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000196","name":"Canada Foundation for Innovation","doi-asserted-by":"publisher","award":["John R. Evans Leaders Fund (CFI JELF) [44548]"],"award-info":[{"award-number":["John R. Evans Leaders Fund (CFI JELF) [44548]"]}],"id":[{"id":"10.13039\/501100000196","id-type":"DOI","asserted-by":"publisher"}]},{"name":"NCOHR Seed Grant"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["J Dent Res"],"abstract":"<jats:p>Intraoral mucosal lesions are difficult to diagnose because they vary widely in appearance and cause. Deep-learning models perform well for oral lesion classification, but limited interpretability restricts clinical use. Counterfactual explanations, which modify image features to test model sensitivity, offer a practical way to improve transparency, yet no counterfactual dataset exists for intraoral images. This study presents COLLECT, the Counterfactual Oral Lesion Library for Explainable Concept Testing, a curated counterfactual reference dataset and evaluation framework for intraoral lesion images, along with a workflow for assessing model interpretability. COLLECT includes 600 manually edited counterfactual images across 4 lesion types: aphthous ulcer, geographic tongue, hairy tongue, and oral squamous cell carcinoma. Images were modified along clinically important concepts such as size, color, and opacity. We assessed the interpretability of common convolutional neural networks using true class probability, predictive entropy, testing with concept activation vectors (TCAV), and attribution scores. Counterfactual edits reduced true class probability and increased entropy, with the strongest effects observed for opacity and color changes. TCAV results showed that early network layers were less responsive to concept changes, while later layers depended more on baseline features. Attribution scores confirmed that opacity changes lowered model confidence most, followed by color and size. Some models showed limited response to lesion removal, indicating reliance on background anatomy rather than lesion-specific cues. COLLECT supports the systematic evaluation of computer vision models under clinically grounded counterfactual conditions and provides a resource for improving transparency in the automated diagnosis of oral lesions.<\/jats:p>","DOI":"10.1177\/00220345261466240","type":"journal-article","created":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T05:53:05Z","timestamp":1784958785000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["COLLECT: A Counterfactual Oral Lesion Library for Trustworthy AI Evaluation"],"prefix":"10.1177","author":[{"given":"K.T.","family":"Tafti","sequence":"first","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Z.","family":"D\u2019Souza","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6628-0854","authenticated-orcid":false,"given":"A.","family":"Ajam","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"P.","family":"Chauvin","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5060-7738","authenticated-orcid":false,"given":"M.","family":"Magalhaes","sequence":"additional","affiliation":[{"name":"Faculty of Dentistry, University of Toronto, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J.","family":"Fuoco","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"F.F.","family":"Sperandio","sequence":"additional","affiliation":[{"name":"College of Dentistry, University of Saskatchewan, Saskatchewan, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Al-Shehri","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A.","family":"Tichy","sequence":"additional","affiliation":[{"name":"Department of Conservative Dentistry and Periodontology, LMU University Hospital, Ludwig-Maximilian University, Munich, Germany"},{"name":"Institute of Dental Medicine, First Faculty of Medicine of the Charles University, Prague, Czech Republic"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4618-8374","authenticated-orcid":false,"given":"F.","family":"Tamimi","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5462-6258","authenticated-orcid":false,"given":"M.","family":"Ducret","sequence":"additional","affiliation":[{"name":"Facult\u00e9 d\u2019Odontologie, Universit\u00e9 Claude Bernard Lyon 1, Universit\u00e9 de Lyon, Lyon, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1223-1669","authenticated-orcid":false,"given":"F.","family":"Schwendicke","sequence":"additional","affiliation":[{"name":"Department of Conservative Dentistry and Periodontology, LMU University Hospital, Ludwig-Maximilian University, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9406-6518","authenticated-orcid":false,"given":"S.","family":"Madathil","sequence":"additional","affiliation":[{"name":"Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,7,24]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"crossref","unstructured":"Adamu MJ et al. 2024. Efficient and accurate brain tumor classification using hybrid MobileNetV2\u2013support vector machine for magnetic resonance imaging diagnostics in neoplasms. Brain Sci. 14(12):1178. https:\/\/doi.org\/10.3390\/brainsci14121178","DOI":"10.3390\/brainsci14121178"},{"key":"e_1_3_2_3_1","first-page":"9525","volume-title":"NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems","author":"Adebayo J","year":"2018","unstructured":"Adebayo J, et al. 2018 Dec 3\u20138. Sanity checks for saliency maps. In: NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems; Montreal, Canada. Curran Associates Inc. p 9525\u20139536."},{"key":"e_1_3_2_4_1","doi-asserted-by":"crossref","unstructured":"Akbar MU Wang W Eklund A. 2024. Beware of diffusion models for synthesizing medical images\u2014a comparison with GANs in terms of memorizing brain MRI and chest X-ray images. Mach Learn Sci Technol. 6:015022. https:\/\/doi.org\/10.1088\/2632-2153\/ad9a3a","DOI":"10.1088\/2632-2153\/ad9a3a"},{"key":"e_1_3_2_5_1","doi-asserted-by":"crossref","unstructured":"Al Rakib AA Billah MM Ahamed AS Haque MI Azad S. 2024. EfficientNet-based model for automated classification of retinal diseases using fundus images. Eur J Comput Sci Inf Technol. 12(8):48\u201361. https:\/\/doi.org\/10.37745\/ejcsit.2013\/vol12n84861","DOI":"10.37745\/ejcsit.2013\/vol12n84861"},{"key":"e_1_3_2_6_1","doi-asserted-by":"crossref","unstructured":"Alabi RO et al. 2021. Machine learning in oral squamous cell carcinoma: current status clinical concerns and prospects for future\u2014a systematic review. Artif Intell Med. 115:102060. https:\/\/doi.org\/10.1016\/j.artmed.2021.102060","DOI":"10.1016\/j.artmed.2021.102060"},{"key":"e_1_3_2_7_1","doi-asserted-by":"crossref","unstructured":"Atad M et al. 2024. Counterfactual explanations for medical image classification and regression using diffusion autoencoder. Mach Learn Biomed Imaging. 2:4862. https:\/\/doi.org\/10.59275\/j.melba.2024-4862","DOI":"10.59275\/j.melba.2024-4862"},{"key":"e_1_3_2_8_1","unstructured":"Besombes C Patel A Madathil SA. 2023. Incorporating expert prior knowledge for oral lesion recognition. ICLR 2023 TinyPapers. OpenReview; [accessed 2026 Jan 20]. https:\/\/openreview.net\/forum?id=XXsjViheWZ"},{"key":"e_1_3_2_9_1","doi-asserted-by":"crossref","unstructured":"Blagec K Kraiger J Fr\u00fchwirt W Samwald M. 2023. Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals. J Biomed Inform. 137:104274. https:\/\/doi.org\/10.1016\/j.jbi.2022.104274","DOI":"10.1016\/j.jbi.2022.104274"},{"key":"e_1_3_2_10_1","doi-asserted-by":"crossref","unstructured":"Castelvecchi D. 2016. Can we open the black box of AI? Nature. 538(7623):20\u201323. https:\/\/doi.org\/10.1038\/538020a","DOI":"10.1038\/538020a"},{"key":"e_1_3_2_11_1","doi-asserted-by":"crossref","unstructured":"Cheng H Lian J Jiao W. 2024. Enhanced MobileNet for skin cancer image classification with fused spatial channel attention mechanism. Sci Rep. 14(1):80087. https:\/\/doi.org\/10.1038\/s41598-024-80087-w","DOI":"10.1038\/s41598-024-80087-w"},{"key":"e_1_3_2_12_1","doi-asserted-by":"crossref","unstructured":"France K Villa A. 2020. Acute oral lesions. Dermatol Clin. 38(4):441\u2013450. https:\/\/doi.org\/10.1016\/j.det.2020.05.005","DOI":"10.1016\/j.det.2020.05.005"},{"key":"e_1_3_2_13_1","doi-asserted-by":"crossref","unstructured":"Ghorbani A Abid A Zou J. 2019. Interpretation of neural networks is fragile. Proc AAAI Conf Artif Intell. 33(1):3681\u20133688. https:\/\/doi.org\/10.1609\/aaai.v33i01.33013681","DOI":"10.1609\/aaai.v33i01.33013681"},{"key":"e_1_3_2_14_1","doi-asserted-by":"crossref","unstructured":"He K Zhang X Ren S Sun J. 2016 Jun 26\u2013Jul 1. Deep residual learning for image recognition. In: Proceedings of the 29th IEEE Conference on Computer Vision Pattern Recognition (CVPR); Las Vegas NV. IEEE. p 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_15_1","doi-asserted-by":"crossref","unstructured":"Irvin J et al. 2019. CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. Proc AAAI Conf Artif Intell. 33(1):590\u2013597. https:\/\/doi.org\/10.1609\/aaai.v33i01.3301590","DOI":"10.1609\/aaai.v33i01.3301590"},{"key":"e_1_3_2_16_1","doi-asserted-by":"crossref","unstructured":"Jia H et al. 2024. Application of convolutional neural networks in medical images: a bibliometric analysis. Quant Imaging Med Surg. 14(5):3501\u20133518. https:\/\/doi.org\/10.21037\/qims-23-1600","DOI":"10.21037\/qims-23-1600"},{"key":"e_1_3_2_17_1","unstructured":"Kim B et al. 2018. Interpretability beyond feature attribution: quantitative testing with concept activation vectors (TCAV). Proc 35th Int Conf Mach Learn. 80:2668\u20132677. https:\/\/proceedings.mlr.press\/v80\/kim18d.html"},{"key":"e_1_3_2_18_1","doi-asserted-by":"crossref","unstructured":"Lang O Traynis I Liu Y. 2025. Explaining counterfactual images. Nat Biomed Eng. 9(3):287\u2013289. https:\/\/doi.org\/10.1038\/s41551-023-01164-5","DOI":"10.1038\/s41551-023-01164-5"},{"key":"e_1_3_2_19_1","doi-asserted-by":"crossref","unstructured":"Liu Z et al. 2022 Jun 19\u201324. A ConvNet for the 2020s. In: Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR); New Orleans LA. IEEE p 11966\u201311976. https:\/\/doi.org\/10.1109\/CVPR52688.2022.01167","DOI":"10.1109\/CVPR52688.2022.01167"},{"key":"e_1_3_2_20_1","doi-asserted-by":"crossref","unstructured":"Mertes S Huber T Weitz K Heimerl A Andr\u00e9 E. 2022. GANterfactual\u2014counterfactual explanations for medical non-experts using generative adversarial learning. Front Artif Intell. 5:825565. https:\/\/doi.org\/10.3389\/frai.2022.825565","DOI":"10.3389\/frai.2022.825565"},{"key":"e_1_3_2_21_1","doi-asserted-by":"crossref","unstructured":"Moons F Vandervieren E. 2025. Measuring agreement among several raters classifying subjects into one or more (hierarchical) categories: a generalization of Fleiss\u2019 kappa. Behav Res Methods. 57(10):287. https:\/\/doi.org\/10.3758\/s13428-025-02746-8","DOI":"10.3758\/s13428-025-02746-8"},{"key":"e_1_3_2_22_1","doi-asserted-by":"crossref","unstructured":"Patel A et al. 2024. Attention-guided convolutional network for bias-mitigated and interpretable oral lesion classification. Sci Rep. 14(1):31700. https:\/\/doi.org\/10.1038\/s41598-024-81724-0","DOI":"10.1038\/s41598-024-81724-0"},{"key":"e_1_3_2_23_1","doi-asserted-by":"crossref","unstructured":"Poeta E Ciravegna G Pastor E Cerquitelli T Baralis E. 2025. Concept-based explainable artificial intelligence: a survey. ACM Comput Surv. https:\/\/doi.org\/10.1145\/3774643","DOI":"10.1145\/3774643"},{"key":"e_1_3_2_24_1","doi-asserted-by":"crossref","unstructured":"Ribeiro MT Singh S Guestrin C. 2016 Aug 13\u201317. \u201cWhy should I trust you?\u201d: explaining the predictions of any classifier. In: Proceedings of KDD '16: The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; San Francisco CA. Association for Computing Machinery. p 1135\u20131144. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_25_1","doi-asserted-by":"crossref","unstructured":"Salahuddin Z Woodruff HC Chatterjee A Lambin P. 2022. Transparency of deep neural networks for medical image analysis: a review of interpretability methods. Comput Biol Med. 140:105111. https:\/\/doi.org\/10.1016\/j.compbiomed.2021.105111","DOI":"10.1016\/j.compbiomed.2021.105111"},{"key":"e_1_3_2_26_1","doi-asserted-by":"crossref","unstructured":"Sandler M Howard A Zhu M Zhmoginov A Chen LC. 2018 Jun 18\u201322. MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition; Salt Lake City UT. IEEE. p 4510\u20134520. https:\/\/doi.org\/10.1109\/CVPR.2018.00474","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_2_27_1","doi-asserted-by":"crossref","unstructured":"Sep\u00falveda-Fontaine SA Amig\u00f3 JM. 2024. Applications of entropy in data analysis and machine learning: a review. Entropy (Basel). 26(12):1126. https:\/\/doi.org\/10.3390\/e26121126","DOI":"10.3390\/e26121126"},{"key":"e_1_3_2_28_1","doi-asserted-by":"crossref","unstructured":"Shamim MZM et al. 2022. Automated detection of oral pre-cancerous tongue lesions using deep learning for early diagnosis of oral cavity cancer. Comput J. 65(1):91\u2013104. https:\/\/doi.org\/10.1093\/comjnl\/bxaa136","DOI":"10.1093\/comjnl\/bxaa136"},{"key":"e_1_3_2_29_1","unstructured":"Simonyan K Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition [preprint]. arXiv. https:\/\/arxiv.org\/abs\/1409.1556"},{"key":"e_1_3_2_30_1","doi-asserted-by":"crossref","unstructured":"Singla S Eslami M Pollack B Wallace S Batmanghelich K. 2023. Explaining the black-box smoothly\u2014a counterfactual approach. Med Image Anal. 84:102721. https:\/\/doi.org\/10.1016\/j.media.2022.102721","DOI":"10.1016\/j.media.2022.102721"},{"key":"e_1_3_2_31_1","doi-asserted-by":"crossref","unstructured":"Sourlos N et al. 2024. Recommendations for the creation of benchmark datasets for reproducible artificial intelligence in radiology. Insights Imaging. 15:248. https:\/\/doi.org\/10.1186\/s13244-024-01833-2","DOI":"10.1186\/s13244-024-01833-2"},{"key":"e_1_3_2_32_1","unstructured":"Tan M Le Q. 2019. EfficientNet: rethinking model scaling for convolutional neural networks. Proc 36th Int Conf Mach Learn. 97:6105\u20136114. https:\/\/proceedings.mlr.press\/v97\/tan19a.html"},{"key":"e_1_3_2_33_1","doi-asserted-by":"crossref","unstructured":"Tanriver G Soluk Tekkesin M Ergen O. 2021. Automated detection and classification of oral lesions using deep learning to detect oral potentially malignant disorders. Cancers (Basel). 13(11):2766. https:\/\/doi.org\/10.3390\/cancers13112766","DOI":"10.3390\/cancers13112766"},{"key":"e_1_3_2_34_1","unstructured":"Tavakoli Tafti K et al. 2026. COLLECT: counterfactual oral lesion library for explainable concept testing [dataset]. Borealis. https:\/\/doi.org\/10.5683\/SP3\/KXYCNY"},{"key":"e_1_3_2_35_1","doi-asserted-by":"crossref","unstructured":"Vellido A. 2019. The importance of interpretability and visualization in machine learning for applications in medicine and health care. Neural Comput Appl. 32(24):18069\u201318083. https:\/\/doi.org\/10.1007\/s00521-019-04051-w","DOI":"10.1007\/s00521-019-04051-w"},{"key":"e_1_3_2_36_1","doi-asserted-by":"crossref","unstructured":"Wang R et al. 2024. Drop the shortcuts: image augmentation improves fairness and decreases AI detection of race and other demographics from medical images. EBioMedicine. 102:105047. https:\/\/doi.org\/10.1016\/j.ebiom.2024.105047","DOI":"10.1016\/j.ebiom.2024.105047"},{"key":"e_1_3_2_37_1","doi-asserted-by":"crossref","unstructured":"West BT. 2022. Linear mixed models: a practical guide using statistical software. Chapman & Hall\/CRC.","DOI":"10.1201\/9781003181064"},{"key":"e_1_3_2_38_1","doi-asserted-by":"crossref","unstructured":"Wolterink JM et al. 2021. Generative adversarial networks: a primer for radiologists. Radiographics. 41(3):840\u2013857. https:\/\/doi.org\/10.1148\/rg.2021200151","DOI":"10.1148\/rg.2021200151"},{"key":"e_1_3_2_39_1","doi-asserted-by":"crossref","unstructured":"Yang F Liu N Du M Hu X. 2021. Generative counterfactuals for neural networks via attribute-informed perturbation. SIGKDD Explor Newsl. 23(1):59\u201368. https:\/\/doi.org\/10.1145\/3468507.3468517","DOI":"10.1145\/3468507.3468517"},{"key":"e_1_3_2_40_1","doi-asserted-by":"crossref","unstructured":"Yun S et al. 2019 Oct 27\u2013Nov 2. CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV); Seoul Korea (South). IEEE. p 6022\u20136031. https:\/\/doi.org\/10.1109\/ICCV.2019.00612","DOI":"10.1109\/ICCV.2019.00612"}],"container-title":["Journal of Dental Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00220345261466240","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00220345261466240","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00220345261466240","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T08:02:25Z","timestamp":1785139345000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00220345261466240"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,24]]},"references-count":39,"alternative-id":["10.1177\/00220345261466240"],"URL":"https:\/\/doi.org\/10.1177\/00220345261466240","relation":{},"ISSN":["0022-0345","1544-0591"],"issn-type":[{"value":"0022-0345","type":"print"},{"value":"1544-0591","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,24]]},"article-number":"00220345261466240"}}