{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T21:27:25Z","timestamp":1773264445075,"version":"3.50.1"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T00:00:00Z","timestamp":1742947200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T00:00:00Z","timestamp":1742947200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008730","name":"Kreftforeningen","doi-asserted-by":"publisher","award":["214931"],"award-info":[{"award-number":["214931"]}],"id":[{"id":"10.13039\/100008730","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Objectives<\/jats:title>\n            <jats:p>To retrospectively evaluate the performance of a CE-marked AI system for identifying breast cancer on screening mammograms. Evidence from large retrospective studies is crucial for planning prospective studies and to further ensure safe implementation.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Materials and methods<\/jats:title>\n            <jats:p>We used data from screening examinations performed from 2004 to 2021 at ten breast centers in BreastScreen Norway. In the standard independent double reading setting, each radiologist scored each breast from 1 (negative) to 5 (high probability of cancer). The AI system assigned each examination an NT and an SN score; the NT score aimed to classify examinations as negative with minimal misclassification while the SN score aimed to classify examinations as positive with high confidence. N70 was defined as being among the 70% with the lowest NT score and P3 was defined as being among the 3% with the highest SN score.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>A total of 1,017,208 screening examinations were included in the study sample. At N70, 1.8% (107\/5977) of the screen-detected and 34.5% (625\/1812) of the interval cancers were defined as negative. Using P3 to define cases as positive, 81.5% (4871\/5977) of the screen-detected and 19.0% (344\/1812) of the interval cancers were defined as positive. Among the screen-detected cancers in N70, 11.2% (12\/107) had an interpretation score\u2009&gt;\u20092 by both radiologists.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>The AI system performed well according to identifying negative cases and cancer cases. Thus, the AI system can be used to reduce workload\u00a0for the radiologists and potentially increase\u00a0the sensitivity\u00a0of mammography.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Key Points<\/jats:title>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Question<\/jats:italic>\n              <\/jats:bold>\n              <jats:italic>Results from large mammography screening samples not used in training AI algorithms are important to consider when planning prospective studies and implementation<\/jats:italic>.<\/jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Findings<\/jats:italic>\n              <\/jats:bold>\n              <jats:italic>More than 80% of the screening-detected cancers were classified as positive by AI when considering 3% of the examinations with the highest AI risk score as positive<\/jats:italic>.<\/jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Clinical relevance<\/jats:italic>\n              <\/jats:bold>\n              <jats:italic>A lack of radiologists is a challenge in mammographic screening. Our findings support other studies that suggest the use of AI to reduce screen-reading workload<\/jats:italic>.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Graphical Abstract<\/jats:title>\n          <\/jats:sec>","DOI":"10.1007\/s00330-025-11521-4","type":"journal-article","created":{"date-parts":[[2025,3,29]],"date-time":"2025-03-29T18:00:14Z","timestamp":1743271214000},"page":"5685-5694","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Retrospective evaluation of a CE-marked AI system, including 1,017,208 mammography screening examinations"],"prefix":"10.1007","volume":"35","author":[{"given":"Tone","family":"Hovda","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marthe","family":"Larsen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marie Burns","family":"Bergan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonas","family":"Gjesvik","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lars A.","family":"Akslen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0178-8939","authenticated-orcid":false,"given":"Solveig","family":"Hofvind","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,3,26]]},"reference":[{"key":"11521_CR1","unstructured":"IARC (2023) Global cancer observatory International agency for research on cancer IARC: global cancer observatory Lyon France. Available via https:\/\/gco.iarc.fr\/"},{"key":"11521_CR2","unstructured":"ECIBC (2023) Recommendations from the European Breast Cancer Guidelines. Available via https:\/\/healthcare-quality.jrc.ec.europa.eu\/ecibc\/european-breast-cancer-guidelines"},{"key":"11521_CR3","doi-asserted-by":"publisher","first-page":"2821","DOI":"10.1007\/s00330-015-3711-6","volume":"25","author":"EG Klompenhouwer","year":"2015","unstructured":"Klompenhouwer EG, Voogd AC, den Heeten GJ et al (2015) Discrepant screening mammography assessments at blinded and non-blinded double reading: impact of arbitration by a third reader on screening outcome. Eur Radiol 25:2821\u20132829","journal-title":"Eur Radiol"},{"key":"11521_CR4","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1148\/radiol.2018171010","volume":"287","author":"S Taylor-Phillips","year":"2018","unstructured":"Taylor-Phillips S, Jenkinson D, Stinton C et al (2018) Double reading in breast cancer screening: cohort evaluation in the CO-OPS trial. Radiology 287:749\u2013757","journal-title":"Radiology"},{"key":"11521_CR5","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1159\/000503715","volume":"14","author":"F Peintinger","year":"2019","unstructured":"Peintinger F (2019) National breast screening programs across Europe. Breast Care 14:354\u2013358","journal-title":"Breast Care"},{"key":"11521_CR6","doi-asserted-by":"publisher","DOI":"10.1038\/s41523-017-0014-x","volume":"3","author":"N Houssami","year":"2017","unstructured":"Houssami N, Hunter K (2017) The epidemiology, radiology and biological characteristics of interval breast cancers in population mammography screening. NPJ Breast Cancer 3:12","journal-title":"NPJ Breast Cancer"},{"key":"11521_CR7","doi-asserted-by":"publisher","unstructured":"Rawson JV, Rubin E, Smetherman D (2024) Short-term strategies for augmenting the national radiologist workforce. AJR Am J Roentgenol. https:\/\/doi.org\/10.2214\/AJR.24.30920","DOI":"10.2214\/AJR.24.30920"},{"key":"11521_CR8","doi-asserted-by":"publisher","first-page":"370","DOI":"10.2214\/AJR.08.1665","volume":"192","author":"P Wing","year":"2009","unstructured":"Wing P, Langelier MH (2009) Workforce shortages in breast imaging: impact on mammography utilization. AJR Am J Roentgenol 192:370\u2013378","journal-title":"AJR Am J Roentgenol"},{"key":"11521_CR9","doi-asserted-by":"publisher","first-page":"110321","DOI":"10.1016\/j.ejrad.2022.110321","volume":"152","author":"D Byng","year":"2022","unstructured":"Byng D, Strauch B, Gnas L et al (2022) AI-based prevention of interval cancers in a national mammography screening program. Eur J Radiol 152:110321","journal-title":"Eur J Radiol"},{"key":"11521_CR10","doi-asserted-by":"publisher","first-page":"S22405","DOI":"10.1117\/1.JMI.10.S2.S22405","volume":"10","author":"K Dembrower","year":"2023","unstructured":"Dembrower K, Salim M, Eklund M et al (2023) Implications for downstream workload based on calibrating an artificial intelligence detection algorithm by standalone-reader or combined-reader sensitivity matching. J Med Imaging 10:S22405","journal-title":"J Med Imaging"},{"key":"11521_CR11","doi-asserted-by":"publisher","unstructured":"Eisemann N, Bunk S, Mukama T et al (2025) Nationwide real-world implementation of AI for cancer detection in population-based mammography screening. Nat Med. https:\/\/doi.org\/10.1038\/s41591-024-03408-6","DOI":"10.1038\/s41591-024-03408-6"},{"key":"11521_CR12","doi-asserted-by":"publisher","first-page":"e803","DOI":"10.1016\/S2589-7500(24)00173-0","volume":"6","author":"ZV Fisches","year":"2024","unstructured":"Fisches ZV, Ball M, Mukama T et al (2024) Strategies for integrating artificial intelligence into mammography screening programmes: a retrospective simulation analysis. Lancet Digit Health 6:e803\u2013e814","journal-title":"Lancet Digit Health"},{"key":"11521_CR13","doi-asserted-by":"publisher","first-page":"e233147","DOI":"10.1148\/radiol.233147","volume":"313","author":"SE Hickman","year":"2024","unstructured":"Hickman SE, Payne NR, Black RT et al (2024) Deep learning algorithms for breast cancer detection in a UK screening cohort: as stand-alone readers and combined with human readers. Radiology 313:e233147","journal-title":"Radiology"},{"key":"11521_CR14","doi-asserted-by":"publisher","first-page":"e230989","DOI":"10.1148\/radiol.230989","volume":"309","author":"M Larsen","year":"2023","unstructured":"Larsen M, Olstad CF, Koch HW et al (2023) AI risk score on screening mammograms preceding breast cancer diagnosis. Radiology 309:e230989","journal-title":"Radiology"},{"key":"11521_CR15","doi-asserted-by":"publisher","first-page":"e230375","DOI":"10.1148\/ryai.230375","volume":"6","author":"M Larsen","year":"2024","unstructured":"Larsen M, Olstad CF, Lee CI et al (2024) Performance of an artificial intelligence system for breast cancer detection on screening mammograms from BreastScreen Norway. Radiol Artif Intell 6:e230375","journal-title":"Radiol Artif Intell"},{"key":"11521_CR16","doi-asserted-by":"publisher","first-page":"e507","DOI":"10.1016\/S2589-7500(22)00070-X","volume":"4","author":"C Leibig","year":"2022","unstructured":"Leibig C, Brehmer M, Bunk S et al (2022) Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis. Lancet Digit Health 4:e507\u2013e519","journal-title":"Lancet Digit Health"},{"key":"11521_CR17","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1016\/S1470-2045(23)00298-X","volume":"24","author":"K L\u00e5ng","year":"2023","unstructured":"L\u00e5ng K, Josefsson V, Larsson AM et al (2023) Artificial intelligence-supported screen reading versus standard double reading in the mammography screening with artificial intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study. Lancet Oncol 24:936\u2013944","journal-title":"Lancet Oncol"},{"key":"11521_CR18","doi-asserted-by":"publisher","first-page":"1581","DOI":"10.1001\/jamaoncol.2020.3321","volume":"6","author":"M Salim","year":"2020","unstructured":"Salim M, W\u00e5hlin E, Dembrower K et al (2020) External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms. JAMA Oncol 6:1581\u20131588","journal-title":"JAMA Oncol"},{"key":"11521_CR19","unstructured":"Lovdata (2001) Regulations on the collection and processing of personal health data in the Cancer Registry of Norway (Cancer Registry Regulations). Available via https:\/\/lovdata.no\/dokument\/SF\/forskrift\/2001-12-21-1477"},{"key":"11521_CR20","unstructured":"Bj\u00f8rnson E, Holen \u00c5S, Sagstad S et al (2022) BreastScreen Norway: 25 years of organized screening. Report no.: ISBN 978-82-93804-03-1. Cancer Registry of Norway, Norway"},{"key":"11521_CR21","unstructured":"Vara (2024) Decision referral: combining strengths of radiologists and AI. Available via https:\/\/www.vara.ai\/decision-referral"},{"key":"11521_CR22","doi-asserted-by":"publisher","first-page":"2206","DOI":"10.1093\/annonc\/mdt303","volume":"24","author":"A Goldhirsch","year":"2013","unstructured":"Goldhirsch A, Winer EP, Coates AS et al (2013) Personalizing the treatment of women with early breast cancer: highlights of the St Gallen International Expert Consensus on the primary therapy of early breast cancer 2013. Ann Oncol 24:2206\u20132223","journal-title":"Ann Oncol"},{"key":"11521_CR23","unstructured":"Sickles E, D\u2019Orsi CJ, Bassett LW et al (2013) ACR BI-RADS\u00ae mammography. In: ACR BI-RADS\u00ae atlas, breast imaging reporting and data system. American College of Radiology, Reston"},{"key":"11521_CR24","doi-asserted-by":"publisher","first-page":"110913","DOI":"10.1016\/j.ejrad.2023.110913","volume":"165","author":"T Hovda","year":"2023","unstructured":"Hovda T, Larsen M, Romundstad L et al (2023) Breast cancer missed at screening; hindsight or mistakes? Eur J Radiol 165:110913","journal-title":"Eur J Radiol"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-025-11521-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-025-11521-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-025-11521-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,6]],"date-time":"2025-09-06T08:55:11Z","timestamp":1757148911000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-025-11521-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,26]]},"references-count":24,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["11521"],"URL":"https:\/\/doi.org\/10.1007\/s00330-025-11521-4","relation":{},"ISSN":["1432-1084"],"issn-type":[{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,26]]},"assertion":[{"value":"9 December 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 January 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 February 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 March 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The scientific guarantor of this publication is Solveig Hofvind.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"No complex statistical methods were necessary for this paper, but M.L. and J.G. have significant statistical expertise.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"Written informed consent was waived by the Institutional Review Board. The data was disclosed with a legal basis in the Cancer Registry Regulations Sections 3\u20131 and the Personal Health Filing System Act section 19a to 19h.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Institutional Review Board approval was obtained, i.e., the study was approved by the Regional Committee for Medical and Health Research Ethics (2018\/13294).","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Numerous studies have been published using screening examinations from BreastScreen Norway. Results concerning AI using subsamples of the sample in this study have been reported in , , , , and .","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Study subjects or cohorts overlap"}},{"order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}