{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T04:57:46Z","timestamp":1781672266777,"version":"3.54.5"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Swedish innovation agency Vinnova"},{"name":"Regional Cancer Centers in Collaboration"},{"name":"Region Stockholm"},{"name":"Swedish Cancer Society"},{"DOI":"10.13039\/501100004359","name":"Swedish Research Council","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100009745","name":"Emil and Wera Cornell Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100009745","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Lillian Sagens and Curt Ericssons Foundation"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>AI cancer detection models require calibration to attain the desired balance between cancer detection rate (CDR) and false positive rate. In this study, we simulate the impact of six types of mismatches between the calibration population and the clinical target population, by creating purposefully non-representative datasets to calibrate AI for clinical settings. Mismatching the acquisition year between healthy and cancer-diagnosed screening participants led to a distortion in CDR between \u22123% to +19%. Mismatching age led to a distortion in CDR between \u22120.2% to +27%. Mismatching breast density distribution led to a distortion in CDR between +1% to 16%. Mismatching mammography vendors lead to a distortion in CDR between \u221232% to + 33%. Mismatches between calibration population and target clinical population lead to clinically important deviations. It is vital for safe clinical AI integration to ensure that important aspects of the calibration population are representative of the target population.<\/jats:p>","DOI":"10.1038\/s41746-025-01623-0","type":"journal-article","created":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T05:40:17Z","timestamp":1746682817000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Simulating mismatch between calibration and target population in AI for mammography the retrospective VAIB study"],"prefix":"10.1038","volume":"8","author":[{"given":"Haiko","family":"Schurz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klara","family":"Solander","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Davida","family":"\u00c5str\u00f6m","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fernando","family":"Coss\u00edo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taeyang","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Magnus","family":"Dustler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claes","family":"Lundstr\u00f6m","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"H\u00e5kan","family":"Gustafsson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sophia","family":"Zackrisson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fredrik","family":"Strand","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,5,8]]},"reference":[{"key":"1623_CR1","doi-asserted-by":"crossref","unstructured":"Dembrower, K. et al. Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Articles Lancet Digit. Health. 2 www.thelancet.com\/ (2020).","DOI":"10.1016\/S2589-7500(20)30185-0"},{"key":"1623_CR2","doi-asserted-by":"publisher","first-page":"1581","DOI":"10.1001\/jamaoncol.2020.3321","volume":"6","author":"M Salim","year":"2020","unstructured":"Salim, M. et al. External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms. JAMA Oncol. 6, 1581\u20131588 (2020).","journal-title":"JAMA Oncol."},{"key":"1623_CR3","doi-asserted-by":"publisher","first-page":"e0265751","DOI":"10.1371\/journal.pone.0265751","volume":"17","author":"D Ueda","year":"2022","unstructured":"Ueda, D. et al. Development and validation of a deep learning model for detection of breast cancers in mammography from multi-institutional datasets. PLoS One 17, e0265751 (2022).","journal-title":"PLoS One"},{"key":"1623_CR4","doi-asserted-by":"publisher","first-page":"e507","DOI":"10.1016\/S2589-7500(22)00070-X","volume":"4","author":"C Leibig","year":"2022","unstructured":"Leibig, C. et al. Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis. Lancet Digit. Health 4, e507\u2013e519 (2022).","journal-title":"Lancet Digit. Health"},{"key":"1623_CR5","doi-asserted-by":"publisher","first-page":"e222639","DOI":"10.1148\/radiol.222639","volume":"307","author":"JH Yoon","year":"2023","unstructured":"Yoon, J. H. et al. Standalone AI for breast cancer detection at screening digital mammography and digital breast tomosynthesis: a systematic review and meta-analysis. Radiology 307, e222639 (2023).","journal-title":"Radiology"},{"key":"1623_CR6","doi-asserted-by":"publisher","first-page":"n1872","DOI":"10.1136\/bmj.n1872","volume":"374","author":"K Freeman","year":"2021","unstructured":"Freeman, K. et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ 374, n1872 (2021).","journal-title":"BMJ"},{"key":"1623_CR7","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1016\/S1470-2045(23)00298-X","volume":"24","author":"K L\u00e5ng","year":"2023","unstructured":"L\u00e5ng, K. et al. Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study. Lancet Oncol. 24, 936\u2013944 (2023).","journal-title":"Lancet Oncol."},{"key":"1623_CR8","doi-asserted-by":"publisher","first-page":"3044","DOI":"10.1038\/s41591-023-02625-9","volume":"29","author":"AY Ng","year":"2023","unstructured":"Ng, A. Y. et al. Prospective implementation of AI-assisted screen reading to improve early detection of breast cancer. Nat. Med. 29, 3044\u20133049 (2023).","journal-title":"Nat. Med."},{"key":"1623_CR9","doi-asserted-by":"publisher","unstructured":"Van Leeuwen, K. G., Schalekamp, S., Rutten, M. J. C. M., Van Ginneken, B. & De Rooij, M. Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. https:\/\/doi.org\/10.1007\/s00330-021-07892-z\/Published.","DOI":"10.1007\/s00330-021-07892-z\/Published"},{"key":"1623_CR10","unstructured":"Artificial Intelligence and Machine Learning (AI\/ML)-Enabled Medical Devices | FDA. https:\/\/www.fda.gov\/medical-devices\/software-medical-device-samd\/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices."},{"key":"1623_CR11","doi-asserted-by":"publisher","first-page":"e703","DOI":"10.1016\/S2589-7500(23)00153-X","volume":"5","author":"K Dembrower","year":"2023","unstructured":"Dembrower, K., Crippa, A., Col\u00f3n, E., Eklund, M. & Strand, F. Artificial intelligence for breast cancer detection in screening mammography in Sweden: a prospective, population-based, paired-reader, non-inferiority study. Lancet Digit. Health 5, e703\u2013e711 (2023).","journal-title":"Lancet Digit. Health"},{"key":"1623_CR12","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1126\/science.adg8538","volume":"383","author":"AM Chekroud","year":"2024","unstructured":"Chekroud, A. M., et al. Illusory generalizability of clinical prediction models. Science. 383, 164\u2013167 (2024).","journal-title":"Science."},{"key":"1623_CR13","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1007\/s11606-020-06280-5","volume":"36","author":"S Ahmed","year":"2021","unstructured":"Ahmed, S. et al. Examining the potential impact of race multiplier utilization in estimated glomerular filtration rate calculation on African-American care outcomes. J. Gen. Intern. Med. 36, 464\u2013471 (2021).","journal-title":"J. Gen. Intern. Med."},{"key":"1623_CR14","doi-asserted-by":"publisher","first-page":"4803","DOI":"10.3390\/cancers14194803","volume":"14","author":"A Gastounioti","year":"2022","unstructured":"Gastounioti, A., et al. External validation of a mammography-derived AI-based risk model in a U.S. breast cancer screening cohort of white and black women. Cancers. 14, 4803 (2022).","journal-title":"Cancers."},{"key":"1623_CR15","doi-asserted-by":"publisher","first-page":"061404","DOI":"10.1117\/1.JMI.10.6.061404","volume":"10","author":"F Coss\u00edo","year":"2023","unstructured":"Coss\u00edo, F., et al. VAI-B: a multicenter platform for the external validation of artificial intelligence algorithms in breast imaging. J. Med. Imaging. 10, 061404 (2023).","journal-title":"J. Med. Imaging."},{"key":"1623_CR16","doi-asserted-by":"publisher","first-page":"W292","DOI":"10.2214\/AJR.10.6049","volume":"198","author":"CM Checka","year":"2012","unstructured":"Checka, C. M., Chun, J. E., Schnabel, F. R., Lee, J. & Toth, H. The relationship of mammographic density and age: Implications for breast cancer screening. Am. J. Roentgenol. 198, W292\u2013W295 (2012).","journal-title":"Am. J. Roentgenol."},{"key":"1623_CR17","doi-asserted-by":"publisher","DOI":"10.1186\/s12880-021-00565-9","volume":"21","author":"Y Ji","year":"2021","unstructured":"Ji, Y. et al. The relationship between breast density, age, and mammographic lesion type among Chinese breast cancer patients from a large clinical dataset. BMC Med. Imaging 21, 43 (2021).","journal-title":"BMC Med. Imaging"},{"key":"1623_CR18","doi-asserted-by":"publisher","first-page":"1806","DOI":"10.1001\/jama.2019.16489","volume":"322","author":"Y Liu","year":"2019","unstructured":"Liu, Y., Chen, P. H. C., Krause, J. & Peng, L. How to read articles that use machine learning: users\u2019 guides to the medical literature. JAMA 322, 1806\u20131816 (2019).","journal-title":"JAMA"},{"key":"1623_CR19","doi-asserted-by":"publisher","unstructured":"Lebovitz, S., Levina, N. & Lifshitz-Assaf, H. Is AI ground truth really true? The dangers of training and evaluating AI tools based on experts\u2019 know-what 1. https:\/\/doi.org\/10.25300\/MISQ\/2021\/16564.","DOI":"10.25300\/MISQ\/2021\/16564"},{"key":"1623_CR20","doi-asserted-by":"publisher","unstructured":"Dinga, R., Penninx, B. W. J. H., Veltman, D. J., Schmaal, L. & Marquand, A. F. Beyond accuracy: measures for assessing machine learning models, pitfalls and guidelines. https:\/\/doi.org\/10.1101\/743138.","DOI":"10.1101\/743138"},{"key":"1623_CR21","doi-asserted-by":"publisher","unstructured":"Gallas, B. D. et al. Evaluating imaging and computer-aided detection and diagnosis devices at the FDA. https:\/\/doi.org\/10.1016\/j.acra.2011.12.016.","DOI":"10.1016\/j.acra.2011.12.016"},{"key":"1623_CR22","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1148\/radiol.220182","volume":"306","author":"SH Park","year":"2023","unstructured":"Park, S. H. et al. Methods for clinical evaluation of artificial intelligence algorithms for medical diagnosis. Radiology 306, 20\u201331 (2023).","journal-title":"Radiology"},{"key":"1623_CR23","doi-asserted-by":"publisher","first-page":"e054005","DOI":"10.1136\/bmjopen-2021-054005","volume":"12","author":"M Luke Marinovich","year":"2022","unstructured":"Luke Marinovich, M. et al. Artificial intelligence (AI) to enhance breast cancer screening: protocol for population-based cohort study of cancer detection. BMJ Open 12, e054005 (2022).","journal-title":"BMJ Open"},{"key":"1623_CR24","doi-asserted-by":"publisher","first-page":"916","DOI":"10.1093\/jnci\/djy222","volume":"111","author":"A Rodriguez-Ruiz","year":"2019","unstructured":"Rodriguez-Ruiz, A. et al. Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J. Natl. Cancer Inst. 111, 916\u2013922 (2019).","journal-title":"J. Natl. Cancer Inst."},{"key":"1623_CR25","doi-asserted-by":"publisher","first-page":"908","DOI":"10.3390\/cancers7020815","volume":"7","author":"A McGuire","year":"2015","unstructured":"McGuire, A., Brown, J. A. L., Malone, C., McLaughlin, R. & Kerin, M. J. Effects of age on the detection and management of breast cancer. Cancers. 7, 908 (2015).","journal-title":"Cancers."},{"key":"1623_CR26","doi-asserted-by":"publisher","first-page":"4287","DOI":"10.3390\/cancers13174287","volume":"13","author":"S \u0141ukasiewicz","year":"2021","unstructured":"\u0141ukasiewicz, S. et al. Breast cancer\u2014epidemiology, risk factors, classification, prognostic markers, and current treatment strategies\u2014an updated review. Cancers. 13, 4287 (2021).","journal-title":"Cancers."},{"key":"1623_CR27","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.breast.2021.05.011","volume":"59","author":"M Lagerlund","year":"2021","unstructured":"Lagerlund, M., \u00c5kesson, A. & Zackrisson, S. Population-based mammography screening attendance in Sweden 2017\u20132018: a cross-sectional register study to assess the impact of sociodemographic factors. Breast 59, 16\u201326 (2021).","journal-title":"Breast"},{"key":"1623_CR28","unstructured":"NHS breast screening: helping you decide\u2014GOV.UK. https:\/\/www.gov.uk\/government\/publications\/breast-screening-helping-women-decide\/nhs-breast-screening-helping-you-decide."},{"key":"1623_CR29","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1148\/radiol.2019190872","volume":"294","author":"K Dembrower","year":"2020","unstructured":"Dembrower, K. et al. Comparison of a deep learning risk score and standard mammographic density score for breast cancer risk prediction. Radiology 294, 265\u2013272 (2020).","journal-title":"Radiology"},{"key":"1623_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s43055-023-01129-3","volume":"54","author":"E Badawy","year":"2023","unstructured":"Badawy, E., ElNaggar, R., Soliman, S. A. M. & Elmesidy, D. S. Performance of AI-aided mammography in breast cancer diagnosis: Does breast density matter? Egyptian J. Radiol. Nucl. Med. 54, 1\u20139 (2023).","journal-title":"Egyptian J. Radiol. Nucl. Med."},{"key":"1623_CR31","doi-asserted-by":"crossref","unstructured":"Bergan, M. B. et al. AI performance by mammographic density in a retrospective cohort study of 99,489 participants in BreastScreen Norway. Eur. Radiol. 68, 1\u201311 (2024).","DOI":"10.1007\/s00330-024-10681-z"},{"key":"1623_CR32","doi-asserted-by":"publisher","DOI":"10.1186\/s13058-024-01821-w","volume":"26","author":"MR Kwon","year":"2024","unstructured":"Kwon, M. R. et al. Screening mammography performance according to breast density: a comparison between radiologists versus standalone intelligence detection. Breast Cancer Res 26, 68 (2024).","journal-title":"Breast Cancer Res"},{"key":"1623_CR33","unstructured":"Tingberg, A. M. et al. AI lesion risk score at different exposure settings. SPIE Digit. Library. 13174, 346\u2013351 (2024)."},{"key":"1623_CR34","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1148\/radiol.210948","volume":"304","author":"AD Lauritzen","year":"2022","unstructured":"Lauritzen, A. D. et al. An artificial intelligence\u2013based mammography screening protocol for breast cancer: outcome and radiologist workload. Radiology 304, 41\u201349 (2022).","journal-title":"Radiology"},{"key":"1623_CR35","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2022.1044496","volume":"12","author":"L Garrucho","year":"2023","unstructured":"Garrucho, L. et al. High-resolution synthesis of high-density breast mammograms: Application to improved fairness in deep learning based mass detection. Front. Oncol. 12, 1044496 (2023).","journal-title":"Front. Oncol."},{"key":"1623_CR36","unstructured":"R Core Team R A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna. References\u2014Scientific Research Publishing. https:\/\/www.scirp.org\/reference\/referencespapers?referenceid=3131254 (2021)."},{"key":"1623_CR37","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"ER DeLong","year":"1988","unstructured":"DeLong, E. R., DeLong, D. M. & Clarke-Pearson, D. L. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44, 837 (1988).","journal-title":"Biometrics"},{"key":"1623_CR38","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1007\/s10278-021-00555-x","volume":"35","author":"SE Lee","year":"2022","unstructured":"Lee, S. E., Son, N. H., Kim, M. H. & Kim, E. K. Mammographic density assessment by artificial intelligence-based computer-assisted diagnosis: a comparison with automated volumetric assessment. J. Digit. Imaging 35, 173\u2013179 (2022).","journal-title":"J. Digit. Imaging"},{"key":"1623_CR39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41392-023-01668-1","volume":"9","author":"J Wang","year":"2024","unstructured":"Wang, J. et al. Progression from ductal carcinoma in situ to invasive breast cancer: molecular features and clinical significance. Signal Transduct. Target. Ther. 9, 1\u201328 (2024).","journal-title":"Signal Transduct. Target. Ther."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01623-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01623-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01623-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T17:04:01Z","timestamp":1746723841000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01623-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,8]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1623"],"URL":"https:\/\/doi.org\/10.1038\/s41746-025-01623-0","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,8]]},"assertion":[{"value":"27 August 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 April 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 May 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"F.S. has received speaker fees from Lunit Inc. and from Pfizer Inc. F.S. and F.C. are major shareholders in ClearScanAI AB. M.D. and S.Z. are patent holders (US patent no. PCT\/EP2014\/057372) and have received speaker fees from Siemens Healthineers. S.Z. has a research agreement with ScreenPoint Medical. C.L. is an employee and shareholder of Sectra AB. The other authors report no conflict of interest.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"259"}}