{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T10:10:39Z","timestamp":1790763039762,"version":"4.1.0"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T00:00:00Z","timestamp":1699833600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T00:00:00Z","timestamp":1699833600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000991","name":"Faculty of Medicine, Nursing and Health Sciences, Monash University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000991","id-type":"DOI","asserted-by":"publisher"}]},{"name":"BreastScreen Australia"},{"DOI":"10.13039\/501100001779","name":"Monash University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001779","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Objectives<\/jats:title>\n                <jats:p>To assess the performance of an artificial intelligence (AI) algorithm in the Australian mammography screening program which routinely uses two independent readers with arbitration of discordant results.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>A total of 7533 prevalent round mammograms from 2017 were available for analysis. The AI program classified mammograms into deciles on the basis of breast cancer (BC) risk. BC diagnoses, including invasive BC (IBC) and ductal carcinoma in situ (DCIS), included those from the prevalent round, interval cancers, and cancers identified in the subsequent screening round two years later. Performance was assessed by sensitivity, specificity, positive and negative predictive values, and the proportion of women recalled by the radiologists and identified as higher risk by AI.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Radiologists identified 54 women with IBC and 13 with DCIS with a recall rate of 9.7%. In contrast, 51 of 54 of the IBCs and 12\/13 cases of DCIS were within the higher AI score group (score 10), a recall equivalent of 10.6% (a difference of 0.9% (CI \u22120.03 to 1.89%, <jats:italic>p\u00a0<\/jats:italic>= 0.06). When IBCs were identified in the 2017 round, interval cancers classified as false negatives or with minimal signs in 2017, and cancers from the 2019 round were combined, the radiologists identified 54\/67 and 59\/67 were in the highest risk AI category (sensitivity 80.6% and 88.06 % respectively, a difference that was not different statistically).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>As the performance of AI was comparable to that of expert radiologists, future AI roles in screening could include replacing one reader and supporting arbitration, reducing workload and false positive results.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Clinical relevance statement<\/jats:title>\n                <jats:p>AI analysis of consecutive prevalent screening mammograms from the Australian BreastScreen program demonstrated the algorithm\u2019s ability to match the cancer detection of experienced radiologists, additionally identifying five interval cancers (false negatives), and the majority of the false positive recalls.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Key Points<\/jats:title>\n                <jats:p><jats:italic>\u2022 The AI program was almost as sensitive as the radiologists in terms of identifying prevalent lesions (51\/54 for invasive breast cancer, 63\/67 when including ductal carcinoma in situ).<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 If selected interval cancers and cancers identified in the subsequent screening round were included, the AI program identified more cancers than the radiologists (59\/67 compared with 54\/67, sensitivity 88.06 % and 80.6% respectively p = 0.24).<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 The high negative predictive value of a score of 1\u20139 would indicate a role for AI as a triage tool to reduce the recall rate (specifically false positives).<\/jats:italic><\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s00330-023-10396-7","type":"journal-article","created":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T06:01:50Z","timestamp":1699855310000},"page":"3947-3957","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Performance of artificial intelligence in 7533 consecutive prevalent screening mammograms from the BreastScreen Australia program"],"prefix":"10.1007","volume":"34","author":[{"given":"John","family":"Waugh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jill","family":"Evans","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miranda","family":"Miocevic","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Darren","family":"Lockie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Parisa","family":"Aminzadeh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anne","family":"Lynch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robin J.","family":"Bell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,11,13]]},"reference":[{"key":"10396_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejrad.2021.109661","volume":"144","author":"MG Wallis","year":"2021","unstructured":"Wallis MG (2021) Artificial intelligence for the real world of breast screening. Eur J Radiol 144:109661","journal-title":"Eur J Radiol"},{"key":"10396_CR2","doi-asserted-by":"crossref","unstructured":"Hickman SE, Woitek R, Le EPV et al (2022) Machine learning for workflow applications in screening mammography: systematic review and meta-analysis. Radiology 302:88\u2013104","DOI":"10.1148\/radiol.2021210391"},{"key":"10396_CR3","doi-asserted-by":"crossref","unstructured":"Freeman K, Geppert J, Stinton C et al (2021) Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ 374:n1872","DOI":"10.1136\/bmj.n1872"},{"key":"10396_CR4","unstructured":"Australian Institute of Health and Welfare (2021) BreastScreen Australia Monitoring Report 2021. https:\/\/www.aihw.gov.au\/reports\/cancer-screening\/breastscreen-australia-monitoring-report-2021"},{"key":"10396_CR5","unstructured":"Australian Institute of Health and Welfare (2022) BreastScreen Australia Monitoring Report 2022. https:\/\/www.aihw.gov.au\/reports\/cancer-screening\/breastscreen-australia-monitoring-report-2022"},{"key":"10396_CR6","unstructured":"Commonwealth of Australia (2021) BreastScreen Australia National Accreditation Handbook. https:\/\/www.health.gov.au\/sites\/default\/files\/documents\/2021\/03\/breastscreen-australia-national-accreditation-handbook"},{"key":"10396_CR7","first-page":"1","volume":"116","author":"L Tabar","year":"1983","unstructured":"Tabar L, Dean PB (1983) Teaching atlas of mammography. Fortschr Geb Rontgenstrahlen Nuklearmed Erganzungsbd 116:1\u2013222","journal-title":"Fortschr Geb Rontgenstrahlen Nuklearmed Erganzungsbd"},{"key":"10396_CR8","doi-asserted-by":"publisher","first-page":"5940","DOI":"10.1007\/s00330-021-07686-3","volume":"31","author":"K Lang","year":"2021","unstructured":"Lang K, Solveig H, Rodriguez-Ruiz A, Andersson I (2021) Can artificial intelligence reduce the interval cancer rate in mammographic screening? Eur Radiol 31:5940\u20135947","journal-title":"Eur Radiol"},{"key":"10396_CR9","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/j.media.2013.10.014","volume":"18","author":"A Bria","year":"2014","unstructured":"Bria A, Karssemeijer N, Tortorella F (2014) Learning from unbalanced data: a cascade-based approach for detecting clustered microcalcifications. Med Image Anal 18:241\u2013252","journal-title":"Med Image Anal"},{"key":"10396_CR10","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1109\/TMI.2009.2028611","volume":"28","author":"R Hupse","year":"2009","unstructured":"Hupse R, Karssemeijer N (2009) Use of normal tissue context in computer-aided detection of masses in mammograms. IEEE Trans Med Imaging 28:2033\u20132041","journal-title":"IEEE Trans Med Imaging"},{"key":"10396_CR11","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1148\/radiol.2018181371","volume":"290","author":"A Rodriguez-Ruiz","year":"2019","unstructured":"Rodriguez-Ruiz A, Krupinski E, Mordang JJ, Schilling K, Heywang-Kobrunner SH, Sechopoulos I et al (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290:305\u2013314","journal-title":"Radiology"},{"key":"10396_CR12","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1038\/s41416-021-01333-w","volume":"125","author":"SE Hickman","year":"2021","unstructured":"Hickman SE, Baxter GC, Gilbert FJ (2021) Adoption of artificial intelligence in breast imaging: evaluation, ethical constraints and limitations. Br J Cancer 125:15\u201322","journal-title":"Br J Cancer"},{"key":"10396_CR13","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1038\/s41523-017-0014-x","volume":"3","author":"N Houssami","year":"2017","unstructured":"Houssami N, Hunter K (2017) The epidemiology, radiology and biological characteristics of interval breast cancers in population mammography screening. NPJ Breast Cancer 3:12","journal-title":"NPJ Breast Cancer"},{"key":"10396_CR14","doi-asserted-by":"publisher","first-page":"e468","DOI":"10.1016\/S2589-7500(20)30185-0","volume":"2","author":"K Dembrower","year":"2020","unstructured":"Dembrower K, Wahlin E, Liu Y, Salim M, Smith K, Lindholm P et al (2020) Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digit Health 2:e468\u2013e474","journal-title":"Lancet Digit Health"},{"key":"10396_CR15","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1038\/s41586-019-1799-6","volume":"577","author":"SM McKinney","year":"2020","unstructured":"McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89\u201394","journal-title":"Nature"},{"key":"10396_CR16","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1007\/s00330-021-08217-w","volume":"32","author":"L Kerschke","year":"2022","unstructured":"Kerschke L, Weigel S, Rodriguez-Ruiz A, Karssemeijer N, Heindel W (2022) Using deep learning to assist readers during the arbitration process: a lesion-based retrospective evaluation of breast cancer screening performance. Eur Radiol 32:842\u2013852","journal-title":"Eur Radiol"},{"key":"10396_CR17","doi-asserted-by":"publisher","first-page":"293","DOI":"10.2214\/AJR.18.20532","volume":"212","author":"EB Mendelson","year":"2019","unstructured":"Mendelson EB (2019) Artificial intelligence in breast imaging: potentials and limitations. AJR Am J Roentgenol 212:293\u2013299","journal-title":"AJR Am J Roentgenol"},{"key":"10396_CR18","doi-asserted-by":"publisher","first-page":"e507","DOI":"10.1016\/S2589-7500(22)00070-X","volume":"4","author":"C Leibig","year":"2022","unstructured":"Leibig C, Brehmer M, Bunk S, Byng D, Pinker K, Umutlu L (2022) Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis. Lancet Digit Health 4:e507\u2013e519","journal-title":"Lancet Digit Health"},{"key":"10396_CR19","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1148\/radiol.2021203555","volume":"300","author":"JL Raya-Povedano","year":"2021","unstructured":"Raya-Povedano JL, Romero-Martin S, Elias-Cabot E, Gubern-Merida A, Rodriguez-Ruiz A, Alvarez-Benito M (2021) AI-based strategies to reduce workload in breast cancer screening with mammography and tomosynthesis: a retrospective evaluation. Radiology 300:57\u201365","journal-title":"Radiology"},{"key":"10396_CR20","doi-asserted-by":"publisher","first-page":"620","DOI":"10.1016\/S0140-6736(22)00235-5","volume":"399","author":"A Aristidou","year":"2022","unstructured":"Aristidou A, Jena R, Topol EJ (2022) Bridging the chasm between AI and clinical implementation. Lancet 399:620","journal-title":"Lancet"},{"key":"10396_CR21","doi-asserted-by":"publisher","first-page":"369","DOI":"10.2214\/AJR.21.27071","volume":"219","author":"LR Lamb","year":"2022","unstructured":"Lamb LR, Lehman CD, Gastounioti A, Conant EF, Bahl M (2022) Artificial intelligence (AI) for screening mammography, from the AJR special series on AI applications. AJR Am J Roentgenol 219:369\u2013380","journal-title":"AJR Am J Roentgenol"},{"key":"10396_CR22","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1186\/s13244-022-01247-y","volume":"13","author":"European Society of R","year":"2022","unstructured":"European Society of R (2022) Current practical experience with artificial intelligence in clinical radiology: a survey of the European Society of Radiology. Insights Imaging 13:107","journal-title":"Insights Imaging"},{"key":"10396_CR23","doi-asserted-by":"publisher","first-page":"4825","DOI":"10.1007\/s00330-019-06186-9","volume":"29","author":"A Rodriguez-Ruiz","year":"2019","unstructured":"Rodriguez-Ruiz A, Lang K, Gubern-Merida A, Teuwen J, Broeders M, Gennaro G et al (2019) Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study. Eur Radiol 29:4825\u20134832","journal-title":"Eur Radiol"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-023-10396-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-023-10396-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-023-10396-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T10:06:49Z","timestamp":1718100409000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-023-10396-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,13]]},"references-count":23,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["10396"],"URL":"https:\/\/doi.org\/10.1007\/s00330-023-10396-7","relation":{},"ISSN":["1432-1084"],"issn-type":[{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,13]]},"assertion":[{"value":"15 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 August 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 September 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 November 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The scientific guarantor of this publication is Prof Robin Bell.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"One of the authors (RJB) has significant statistical expertise.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"Written informed consent was not required for this study because all women who are screened through BreastScreen sign a form giving permission for their test results to be used for research purposes.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Institutional Review Board approval was obtained (Monash Health Human Research Ethics Committee Low Risk Panel Monash Health (Monash Health Ref: RES-20-0000-166L-61177)).","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"No AI data on the study subjects have been previously reported.","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Study subjects or cohorts overlap"}},{"value":"\u2022 retrospective\u2022 observational\u2022 performed at one institution","order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}