{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T00:24:52Z","timestamp":1782951892590,"version":"3.54.5"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,9,7]],"date-time":"2024-09-07T00:00:00Z","timestamp":1725667200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,7]],"date-time":"2024-09-07T00:00:00Z","timestamp":1725667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Distributed collaborative learning is a promising approach for building predictive models for privacy-sensitive biomedical images. Here, several data owners (clients) train a joint model without sharing their original data. However, concealed systematic biases can compromise model performance and fairness. This study presents MyThisYourThat (MyTH) approach, which adapts an interpretable prototypical part learning network to a distributed setting, enabling each client to visualize feature differences learned by others on their own image: comparing one client\u2019s 'This\u2019 with others\u2019 'That\u2019. Our setting demonstrates four clients collaboratively training two diagnostic classifiers on a benchmark X-ray dataset. Without data bias, the global model reaches 74.14% balanced accuracy for cardiomegaly and 74.08% for pleural effusion. We show that with systematic visual bias in one client, the performance of global models drops to near-random. We demonstrate how differences between local and global prototypes reveal biases and allow their visualization on each client\u2019s data without compromising privacy.<\/jats:p>","DOI":"10.1038\/s41746-024-01226-1","type":"journal-article","created":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T21:02:01Z","timestamp":1725656521000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["MyThisYourThat for interpretable identification of systematic bias in federated learning for biomedical images"],"prefix":"10.1038","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0951-9576","authenticated-orcid":false,"given":"Klavdiia","family":"Naumova","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9425-1163","authenticated-orcid":false,"given":"Arnout","family":"Devos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai Praneeth","family":"Karimireddy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Jaggi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mary-Anne","family":"Hartley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,9,7]]},"reference":[{"key":"1226_CR1","doi-asserted-by":"crossref","unstructured":"Piccialli, F., Di Somma, V., Giampaolo, F., Cuomo, S. & Fortino, G. A survey on deep learning in medicine: Why, how and when? Inf. Fusion 66, 111\u2013137 (2021).","DOI":"10.1016\/j.inffus.2020.09.006"},{"key":"1226_CR2","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","volume":"19","author":"D Shen","year":"2017","unstructured":"Shen, D., Wu, G. & Suk, H.-I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221\u2013248 (2017).","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"1226_CR3","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0301-z","volume":"3","author":"I Landi","year":"2020","unstructured":"Landi, I. et al. Deep representation learning of electronic health records to unlock patient stratification at scale. npj Digit. Med. 3, 96 (2020).","journal-title":"npj Digit. Med."},{"key":"1226_CR4","doi-asserted-by":"crossref","unstructured":"Barnett, A. J. et al. Interpretable deep learning models for better clinician-AI communication in clinical mammography. In Proc. of SPIE Medical Imaging 2022: Image Perception, Observer Performance, and Technology Assessment, SPIE Digital Library (eds. Mello-Thoms, C. R. & Taylor-Phillips, S.), 12035, 1203507 (2022).","DOI":"10.1117\/12.2612372"},{"key":"1226_CR5","unstructured":"McMahan, H. B., Moore, E., Ramage, D., Hampson, S. & Arcas, B. A. Communication-efficient learning of deep networks from decentralized data. In Proc. of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) (ed. Lawrence, N.), 54, 1273\u20131282 (JMLR: W&CP, 2017)."},{"key":"1226_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3453476","volume":"55","author":"DC Nguyen","year":"2022","unstructured":"Nguyen, D. C. et al. Federated learning for smart healthcare: a survey. ACM Comput. Surv. 55, 1\u201337 (2022).","journal-title":"ACM Comput. Surv."},{"key":"1226_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-00323-1","volume":"3","author":"N Rieke","year":"2020","unstructured":"Rieke, N. et al. The future of digital health with federated learning. npj Digit. Med. 3, 1\u20137 (2020).","journal-title":"npj Digit. Med."},{"key":"1226_CR8","doi-asserted-by":"publisher","first-page":"1532","DOI":"10.3390\/diagnostics13091532","volume":"13","author":"S Nazir","year":"2023","unstructured":"Nazir, S. & Kaleem, M. Federated learning for medical image analysis with deep neural networks. Diagnostics 13, 1532 (2023).","journal-title":"Diagnostics"},{"key":"1226_CR9","doi-asserted-by":"publisher","first-page":"3779","DOI":"10.1007\/s11063-022-11014-1","volume":"55","author":"M Islam","year":"2023","unstructured":"Islam, M., Reza, M. T., Kaosar, M. & Parvez, M. Z. Effectiveness of federated learning and CNN ensemble architectures for identifying brain tumors using MRI images. Neural Process. Lett. 55, 3779\u20133809 (2023).","journal-title":"Neural Process. Lett."},{"key":"1226_CR10","doi-asserted-by":"crossref","unstructured":"Ribeiro, M. T., Singh, S. & Guestrin, C. \u201cWhy should I trust you?\": explaining the predictions of any classifier. In Proc. of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery (eds. Krishnapuram, B., Shah, M., Smola, A. J., Aggarwal, C., Shen D. & Rastogi, R.), 1135\u20131144 (2016).","DOI":"10.1145\/2939672.2939778"},{"key":"1226_CR11","unstructured":"Lundberg, S. & Lee, S.-I. A unified approach to interpreting model predictions. In Proc. of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc. (eds. von Luxburg, U., Guyon, I., Bengio, S., Wallach H. & Fergus, R.), 4768\u20134777 (2017)."},{"key":"1226_CR12","doi-asserted-by":"crossref","unstructured":"Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. J. Comput. Vis. 128, 336\u2013359 (2019).","DOI":"10.1007\/s11263-019-01228-7"},{"key":"1226_CR13","unstructured":"Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: visualising image classification models and saliency maps. In Workshop at International Conference on Learning Representations (2014)."},{"key":"1226_CR14","doi-asserted-by":"publisher","first-page":"52","DOI":"10.3390\/jimaging6060052","volume":"6","author":"A Singh","year":"2020","unstructured":"Singh, A., Sengupta, S. & Lakshminarayanan, V. Explainable deep learning models in medical image analysis. J. Imaging 6, 52 (2020).","journal-title":"J. Imaging"},{"key":"1226_CR15","first-page":"3145","volume":"70","author":"A Shrikumar","year":"2017","unstructured":"Shrikumar, A., Greenside, P. & Kundaje, A. Learning important features through propagating activation differences. PMLR 70, 3145\u20133153 (2017).","journal-title":"PMLR"},{"key":"1226_CR16","doi-asserted-by":"crossref","unstructured":"Chen, H., Lundberg, S. & Lee, S.-I. Explaining Models by Propagating Shapley Values of Local Components 261\u2013270 (Springer International Publishing, Cham, 2021).","DOI":"10.1007\/978-3-030-53352-6_24"},{"key":"1226_CR17","doi-asserted-by":"crossref","unstructured":"Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K. & M\u00fcller, K.-R. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (Springer, 2019).","DOI":"10.1007\/978-3-030-28954-6"},{"key":"1226_CR18","unstructured":"Molnar, C. Interpretable Machine Learning (Independently published, 2023)."},{"key":"1226_CR19","unstructured":"Adebayo, J. et al. Sanity checks for saliency maps. Adv. Neural Inf. Process. Syst. 31, 9505\u20139515 (2018)."},{"key":"1226_CR20","doi-asserted-by":"crossref","unstructured":"Rudin, C. et al. Interpretable machine learning: fundamental principles and 10 grand challenges. Stat. Surv. 16, 1\u201385 (2022).","DOI":"10.1214\/21-SS133"},{"key":"1226_CR21","unstructured":"Sun, S., Woerner, S., Maier, A., Koch, L. M. & Baumgartner, C. F. Inherently interpretable multi-label classification using class-specific counterfactuals. Proc. Mach. Learn. Res. 227, 937\u2013956 (2023)."},{"key":"1226_CR22","unstructured":"Chen, C. et al. This looks like that: deep learning for interpretable image recognition. In Proc. of the 33rd International Conference on Neural Information Processing Systems, Curran Associates Inc. (eds. Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch\u00e9-Buc, F., Fox E. & Garnett, R.), 8930\u20138941 (2019)."},{"key":"1226_CR23","doi-asserted-by":"publisher","first-page":"1067","DOI":"10.1038\/s42256-021-00423-x","volume":"3","author":"A Barnett","year":"2021","unstructured":"Barnett, A. et al. A case-based interpretable deep learning model for classification of mass lesions in digital mammography. Nat. Mach. Intell. 3, 1067\u20131070 (2021).","journal-title":"Nat. Mach. Intell."},{"key":"1226_CR24","doi-asserted-by":"crossref","unstructured":"Hase, P., Chen, C., Li, O. & Rudin, C. Interpretable image recognition with hierarchical prototypes. In Proc. of the 7th AAAI Conference on Human Computation & Crowdsourcing, PKP:PS (eds. Law, E. & Vaughan, J. W.), 7, 32\u201340 (2019).","DOI":"10.1609\/hcomp.v7i1.5265"},{"key":"1226_CR25","doi-asserted-by":"crossref","unstructured":"Irvin, J. et al. Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In Proc. of the 33rd AAAI Conference on Artificial Intelligence (AAAI-19) (eds. Van Hentenryck, P. & Zhou, Z.-H.), 33, 590\u2013597 (AAAI Press, 2019).","DOI":"10.1609\/aaai.v33i01.3301590"},{"key":"1226_CR26","doi-asserted-by":"crossref","unstructured":"Jim\u00e9nez-S\u00e1nchez, A., Juodelye, D., Chamberlain, B. & Cheplygina, V. Detecting shortcuts in medical images\u2014a case study in chest X-rays. In Proc. of the IEEE 20th International Symposium on Biomedical Imaging (ISBI), (eds. Salvado, O. & Rittner, L.), 1\u20135 (IEEE, 2023).","DOI":"10.1109\/ISBI53787.2023.10230572"},{"key":"1226_CR27","doi-asserted-by":"publisher","first-page":"119898","DOI":"10.1016\/j.ins.2023.119898","volume":"655","author":"JD Ser","year":"2024","unstructured":"Ser, J. D. et al. On generating trustworthy counterfactual explanations. Inf. Sci. 655, 119898 (2024).","journal-title":"Inf. Sci."},{"key":"1226_CR28","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (ed. O\u2019Conner, L.), 2261\u20132269 (IEEE, 2017).","DOI":"10.1109\/CVPR.2017.243"},{"key":"1226_CR29","doi-asserted-by":"crossref","unstructured":"Deng, J. et al. ImageNet: a large-scale hierarchical image database. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (eds. Essa, I., Kang S. B. & Pollefeys, M.), 248\u2013255 (IEEE, 2009).","DOI":"10.1109\/CVPR.2009.5206848"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01226-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01226-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01226-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,8]],"date-time":"2024-09-08T13:02:40Z","timestamp":1725800560000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01226-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,7]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["1226"],"URL":"https:\/\/doi.org\/10.1038\/s41746-024-01226-1","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3910714\/v1","asserted-by":"object"}]},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,7]]},"assertion":[{"value":"30 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 September 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"238"}}