{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T21:23:01Z","timestamp":1792790581403,"version":"4.5.0"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T00:00:00Z","timestamp":1772668800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T00:00:00Z","timestamp":1772668800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>In breast cancer screening, determining the optimal balance between the number of screen-detected cancer cases and false-positive recalls is essential. This study explored the relationship between these indicators for the Dutch Digital Mammography Screening Program and aimed to determine the optimal recall rate.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>From March to June 2019, 21 screening radiologists provided continuous Probability-of-Malignancy (PoM) scores during their standard reading sessions. Scores ranged from \u2018no suspicion\u2019 (PoM\u2009=\u2009\u2212100) to \u2018highest suspicion\u2019 (PoM\u2009=\u2009100). Follow-up data were obtained in June 2024 and included recall decisions after double reading, outcomes of further assessments (false positive or screen-detected cancer), and interval cancer diagnoses. Recall\u2013detection and receiver operating characteristic (ROC) curves were constructed for each reader and averaged to obtain summary curves, with error bars computed from hierarchical bootstrapping of cases within readers (1000 resamples). The overall screening performance was quantified using the area under the ROC curve (AUC).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The final dataset comprised 40,829 screening records with 315 cancer cases. The overall recall rate was 2.33%, and the cancer detection rate after double reading was 6.4 per 1000 screens. Radiologist performance was high (AUC\u2009=\u20090.91). Moving the operating point results in either substantially lower cancer detection or relatively little gain.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This prospective study identified the trade-off between unconditional recall and detection rates, as well as conditional sensitivity and specificity. We found that Dutch screening radiologists perform at a high level and operate at a point that seems to optimize the false-positive recall and cancer detection rate trade-off.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Key Points<\/jats:title>\n                    <jats:p>\n                      <jats:bold>\n                        <jats:italic>Question<\/jats:italic>\n                      <\/jats:bold>\n                      <jats:italic>Breast cancer screening requires a good balance between detection and false-positive rate. However, the interrelationship between these rates, and thus the optimal recall, is unknown.<\/jats:italic>\n                    <\/jats:p>\n                    <jats:p>\n                      <jats:bold>\n                        <jats:italic>Findings<\/jats:italic>\n                      <\/jats:bold>\n                      <jats:italic>Overall, the Dutch screening radiologists performed with high accuracy, and the current operating point optimizes the trade-off between cancer detection and false-positive recalls<\/jats:italic>\n                      .\n                    <\/jats:p>\n                    <jats:p>\n                      <jats:bold>\n                        <jats:italic>Clinical relevance<\/jats:italic>\n                      <\/jats:bold>\n                      <jats:italic>ROCS provides an efficient method to evaluate performance and determine target values based on data from screening practice. This method applies to various screening programs. Screening evaluation is essential for achieving and maintaining a positive benefit-to-harm ratio for participants<\/jats:italic>\n                      .\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00330-026-12370-5","type":"journal-article","created":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T23:02:11Z","timestamp":1772665331000},"page":"5595-5603","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Finding the optimal recall rate in breast cancer screening: results from the ROCS study"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5850-8814","authenticated-orcid":false,"given":"Dani\u00eblle","family":"van der Waal","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Craig K.","family":"Abbey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Tetteroo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tanya D.","family":"Geertse","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maartje J. A.","family":"Smid-Geirnaerdt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ioannis","family":"Sechopoulos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mireille J. M.","family":"Broeders","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,3,5]]},"reference":[{"key":"12370_CR1","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1016\/j.ejca.2019.12.010","volume":"127","author":"N Zielonke","year":"2020","unstructured":"Zielonke N, Gini A, Jansen EEL et al (2020) Evidence for reducing cancer-specific mortality due to screening for breast cancer in Europe: a systematic review. Eur J Cancer 127:191\u2013206. https:\/\/doi.org\/10.1016\/j.ejca.2019.12.010","journal-title":"Eur J Cancer"},{"key":"12370_CR2","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1002\/ijc.34530","volume":"153","author":"LM Kregting","year":"2023","unstructured":"Kregting LM, van Ravesteyn NT, Chootipongchaivat S et al (2023) Cumulative risks of false positive recall and screen-detected breast cancer after multiple screening examinations. Int J Cancer 153:312\u2013319. https:\/\/doi.org\/10.1002\/ijc.34530","journal-title":"Int J Cancer"},{"key":"12370_CR3","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1038\/s41416-019-0524-4","volume":"121","author":"H Long","year":"2019","unstructured":"Long H, Brooks JM, Harvie M, Maxwell A, French DP (2019) How do women experience a false-positive test result from breast screening? A systematic review and thematic synthesis of qualitative studies. Br J Cancer 121:351\u2013358. https:\/\/doi.org\/10.1038\/s41416-019-0524-4","journal-title":"Br J Cancer"},{"key":"12370_CR4","doi-asserted-by":"publisher","first-page":"1297","DOI":"10.7326\/M24-0123","volume":"177","author":"DL Miglioretti","year":"2024","unstructured":"Miglioretti DL, Abraham L, Sprague BL et al (2024) Association between false-positive results and return to screening mammography in the Breast Cancer Surveillance Consortium Cohort. Ann Intern Med 177:1297\u20131307. https:\/\/doi.org\/10.7326\/M24-0123","journal-title":"Ann Intern Med"},{"key":"12370_CR5","doi-asserted-by":"publisher","first-page":"103702","DOI":"10.1016\/j.breast.2024.103702","volume":"74","author":"AC Voogd","year":"2024","unstructured":"Voogd AC, Molnar Z, Nederend J, Schipper RJ, Strobbe LJA, Duijm LEM (2024) Predictors of re-attendance at biennial screening mammography following a false positive referral: a study among women in the south of the Netherlands. Breast 74:103702. https:\/\/doi.org\/10.1016\/j.breast.2024.103702","journal-title":"Breast"},{"key":"12370_CR6","doi-asserted-by":"publisher","first-page":"1855","DOI":"10.1002\/ijc.32968","volume":"147","author":"P Armaroli","year":"2020","unstructured":"Armaroli P, Riggi E, Basu P et al (2020) Performance indicators in breast cancer screening in the European Union: a comparison across countries of screen positivity and detection rates. Int J Cancer 147:1855\u20131863. https:\/\/doi.org\/10.1002\/ijc.32968","journal-title":"Int J Cancer"},{"key":"#cr-split#-12370_CR7.1","unstructured":"Landelijk Evaluatie Team voor bevolkingsonderzoek naar Borstkanker (LETB) [National Evaluation Team for Breast Cancer Screening (NETB)] (2023) Landelijke evaluatie van bevolkingsonderzoek naar borstkanker in Nederland: het vijftiende evaluatierapport"},{"key":"#cr-split#-12370_CR7.2","unstructured":"(XV) [National evaluation of population-based breast cancer screening in the Netherlands: the fifteenth evaluation report (XV)]. Radboudumc, Erasmus MC, Nijmegen, Rotterdam. Available via https:\/\/www.rivm.nl\/sites\/default\/files\/2023-08\/LETB%20-%20Resultaten%20bevolkingsonderzoek%20borstkanker%202023%20%28rapport%20XV%29.pdf. Accessed 21 Oct 2025"},{"key":"12370_CR8","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1093\/jnci\/dji131","volume":"97","author":"JDM Otten","year":"2005","unstructured":"Otten JDM, Karssemeijer N, Hendriks JHCL et al (2005) Effect of recall rate on earlier screen detection of breast cancers based on the Dutch performance indicators. J Natl Cancer Inst 97:748\u2013754. https:\/\/doi.org\/10.1093\/jnci\/dji131","journal-title":"J Natl Cancer Inst"},{"key":"12370_CR9","doi-asserted-by":"publisher","first-page":"2067","DOI":"10.1007\/s00330-010-1786-7","volume":"20","author":"AMJ Bluekens","year":"2010","unstructured":"Bluekens AMJ, Karssemeijer N, Beijerinck D et al (2010) Consequences of digital mammography in population-based breast cancer screening: initial changes and long-term impact on referral rates. Eur Radiol 20:2067\u20132073. https:\/\/doi.org\/10.1007\/s00330-010-1786-7","journal-title":"Eur Radiol"},{"key":"12370_CR10","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1093\/eurpub\/ckr170","volume":"22","author":"JM Timmers","year":"2012","unstructured":"Timmers JM, Den Heeten GJ, Adang EM, Otten JD, Verbeek AL, Broeders MJ (2012) Dutch digital breast cancer screening: implications for breast cancer care. Eur J Public Health 22:925\u2013929. https:\/\/doi.org\/10.1093\/eurpub\/ckr170","journal-title":"Eur J Public Health"},{"key":"12370_CR11","doi-asserted-by":"publisher","first-page":"7463","DOI":"10.1007\/s00330-022-08820-5","volume":"32","author":"I Sechopoulos","year":"2022","unstructured":"Sechopoulos I, Abbey CK, van der Waal D et al (2022) Evaluation of reader performance during interpretation of breast cancer screening: the recall and detection Of breast Cancer in Screening (ROCS) trial study design. Eur Radiol 32:7463\u20137469. https:\/\/doi.org\/10.1007\/s00330-022-08820-5","journal-title":"Eur Radiol"},{"key":"12370_CR12","doi-asserted-by":"publisher","first-page":"015501","DOI":"10.1117\/1.JMI.6.1.015501","volume":"6","author":"BD Gallas","year":"2019","unstructured":"Gallas BD, Chen W, Cole E et al (2019) Impact of prevalence and case distribution in lab-based diagnostic imaging studies. J Med Imaging 6:015501. https:\/\/doi.org\/10.1117\/1.JMI.6.1.015501","journal-title":"J Med Imaging"},{"key":"12370_CR13","unstructured":"Rijksinstituut voor Volksgezondheid en Milieu (RIVM) [National Institute for Public Health and the Environment] (2017) Indicatoren voor Bevolkingsonderzoek naar Borstkanker 2017 (versie 2.2) [Indicators for population-based breast cancer screening 2017 (version 2.2)]. RIVM, Bilthoven. Available via https:\/\/www.rivm.nl\/sites\/default\/files\/2018-11\/BK_indicatoren_2017%20versie%202.2.pdf. Accessed 21 Oct 2025"},{"key":"12370_CR14","doi-asserted-by":"publisher","first-page":"208","DOI":"10.2214\/AJR.15.15987","volume":"208","author":"P Grabler","year":"2017","unstructured":"Grabler P, Sighoko D, Wang L, Allgood K, Ansell D (2017) Recall and cancer detection rates for screening mammography: finding the sweet spot. AJR Am J Roentgenol 208:208\u2013213. https:\/\/doi.org\/10.2214\/AJR.15.15987","journal-title":"AJR Am J Roentgenol"},{"key":"12370_CR15","doi-asserted-by":"publisher","first-page":"894","DOI":"10.2214\/AJR.19.22429","volume":"216","author":"GH Rauscher","year":"2021","unstructured":"Rauscher GH, Murphy AM, Qiu Q et al (2021) The \u201cSweet Spot\u201d revisited: optimal recall rates for cancer detection with 2D and 3D digital screening mammography in the Metro Chicago Breast Cancer Registry. AJR Am J Roentgenol 216:894\u2013902. https:\/\/doi.org\/10.2214\/AJR.19.22429","journal-title":"AJR Am J Roentgenol"},{"key":"12370_CR16","doi-asserted-by":"publisher","first-page":"1023","DOI":"10.2214\/AJR.21.25738","volume":"217","author":"RG Barr","year":"2021","unstructured":"Barr RG (2021) Your \u201cSweet Spot\u201d may not be so sweet for me. AJR Am J Roentgenol 217:1023. https:\/\/doi.org\/10.2214\/AJR.21.25738","journal-title":"AJR Am J Roentgenol"},{"key":"12370_CR17","doi-asserted-by":"publisher","first-page":"3812","DOI":"10.1007\/s00330-018-5957-2","volume":"29","author":"RG Blanks","year":"2019","unstructured":"Blanks RG, Given-Wilson RM, Cohen SL, Patnick J, Alison RJ, Wallis MG (2019) An analysis of 11.3 million screening tests examining the association between recall and cancer detection rates in the English NHS breast cancer screening programme. Eur Radiol 29:3812\u20133819. https:\/\/doi.org\/10.1007\/s00330-018-5957-2","journal-title":"Eur Radiol"},{"key":"12370_CR18","unstructured":"NHS Breast Screening Programme Screening standards valid for data collected from 1 April 2021 (2024) BSP-S07: referral: rate of referral to assessment. NHS, London. Available via https:\/\/www.gov.uk\/government\/publications\/breast-screening-consolidated-programme-standards\/nhs-breast-screening-programme-screening-standards-valid-for-data-collected-from-1-april-2021. Accessed 21 Oct 2025"},{"key":"12370_CR19","unstructured":"European Commission (2006) European guidelines for quality assurance in breast cancer screening and diagnosis, 4th edn. In Perry N, Broeders M, de Wolf C et al (eds) Luxembourg: Office for Official Publications of the European Communities. Available via https:\/\/screening.iarc.fr\/doc\/ND7306954ENC_002.pdf. Accessed 21 Oct 2025"},{"key":"12370_CR20","unstructured":"European Commission Initiative on Breast Cancer (ECIBC) (2025) European guidelines on breast cancer screening and diagnosis. Available via https:\/\/cancer-screening-and-care.jrc.ec.europa.eu\/en\/ecibc\/european-breast-cancer-guidelines. Accessed 11 Dec 2025"},{"key":"12370_CR21","doi-asserted-by":"publisher","first-page":"e175","DOI":"10.1016\/S2589-7500(24)00267-X","volume":"7","author":"V Hernstr\u00f6m","year":"2025","unstructured":"Hernstr\u00f6m V, Josefsson V, Sartor H et al (2025) Screening performance and characteristics of breast cancer detected in the Mammography Screening with Artificial Intelligence trial (MASAI): a randomised, controlled, parallel-group, non-inferiority, single-blinded, screening accuracy study. Lancet Digit Health 7:e175\u2013e183. https:\/\/doi.org\/10.1016\/S2589-7500(24)00267-X","journal-title":"Lancet Digit Health"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-026-12370-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-026-12370-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-026-12370-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T10:17:18Z","timestamp":1781864238000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-026-12370-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,5]]},"references-count":22,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["12370"],"URL":"https:\/\/doi.org\/10.1007\/s00330-026-12370-5","relation":{},"ISSN":["1432-1084"],"issn-type":[{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,5]]},"assertion":[{"value":"22 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2026","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 January 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 March 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The scientific guarantor of this publication is Dani\u00eblle van der Waal, PhD.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare relationships with the following companies: I.S. reports receiving grants or contracts from Siemens Healthcare, Canon Medical Systems, ScreenPoint Medical, Volpara Solutions, Sectra Benelux, Lunit, and iCAD, lecture fees from Siemens Healthcare and Canon Medical Systems, and meeting support from Siemens Healthcare and Canon Medical Systems. I.S. also serves on a Scientific Advisory Board for Koning, holds stock options in Koning, and is a member of the Radiology editorial board. M.J.M.B. reports receiving grants or contracts from ScreenPoint Medical, Sectra Benelux, Hologic, Volpara Solutions, Lunit, and iCAD, speaker fees from Hologic and Siemens Healthcare, and travel support from Hologic and Siemens Healthcare. All other authors have nothing to disclose.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"One of the authors, Craig Abbey, PhD, has significant statistical expertise.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"In the Dutch Breast Cancer Screening Program, screening participants automatically consent to the use of their data for research purposes, unless they opt out.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Ethical approval was waived by the local ethics committee of the Radboud University Medical Center (CMO Radboudumc; file number: 2018-4991, confirmation received on February 13, 2019).","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"The design of this cohort study was previously published in European Radiology (see reference [\n                      \n                      ]). The number of initial inclusions is mentioned there, but no further results are presented. Apart from that, the study subjects have not been previously reported.","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Study subjects or cohorts overlap"}},{"order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}