{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:19:59Z","timestamp":1763443199839,"version":"3.45.0"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T00:00:00Z","timestamp":1763424000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The rapid integration of artificial intelligence (AI) into mammography necessitates robust quality control methods. The lack of standardized methods for establishing decision thresholds on the Receiver Operating Characteristic (ROC) curves makes it challenging to judge the AI performance. This study aims to develop a method for determining the decision threshold for AI in screening mammography to ensure the widest possible population of women with a breast pathology is diagnosed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Three AI models were retrospectively evaluated using a dataset of digital mammograms. The dataset consisted of screening mammography examinations obtained from 663,606 patients over the age of 40. Our method estimates the decision threshold using a novel approach to net benefit (NB) analysis. Our approach to setting the cutoff threshold was compared with the threshold determined by Youden's index using McNemar's test.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Replacing the Youden index with our method across three AI models, resulted in a threefold reduction in false-positive rates, twofold reduction in false-negative rates, and twofold increase in true-positive rates. Thus, the sensitivity at the cutoff threshold determined by NB increased to 99% (maximum) compared to the sensitivity determined by Youden's index threshold (72% maximum). Correspondingly, the specificity when using our method decreased to 48% (minimum), compared to 75% (minimum) with the Youden's index method.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We propose using AI as the initial reader together with our novel method for determining the decision threshold in screening with double reading. This approach enhances the AI sensitivity and improves timely breast cancer diagnosis.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdata.2025.1690955","type":"journal-article","created":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:15:40Z","timestamp":1763442940000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Implementation of a net benefit parameter in ROC curve decision thresholds for AI-powered mammography screening"],"prefix":"10.3389","volume":"8","author":[{"given":"Anastasia Petrovna","family":"Pamova","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuriy Aleksandrovich","family":"Vasilev","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tatyana Mikhaylovna","family":"Bobrovskaya","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anton Vyacheslavovich","family":"Vladzimirskyy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Olga Vasilyevna","family":"Omelyanskaya","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kirill Mikhailovich","family":"Arzamasov","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,11,18]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"117","DOI":"10.17116\/profmed202326061117","article-title":"The use of computer vision for the mammography preventive research","volume":"26","author":"Arzamasov","year":"2023","journal-title":"Profilakticheskaya meditsina"},{"key":"B2","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1148\/radiol.13130724","article-title":"Breast cancer detected with screening US: Reasons for nondetection at mammography","volume":"270","author":"Bae","year":"2014","journal-title":"Radiology"},{"key":"B3","doi-asserted-by":"publisher","first-page":"482","DOI":"10.1177\/1403494819845565","article-title":"Productivity losses associated with premature mortality due to cancer in Russia: a population-wide study covering 2001\u20132030","volume":"47","author":"Barchuk","year":"2019","journal-title":"Scand. J. Public Health"},{"key":"B4","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1148\/radiol.2020192641","article-title":"Cancer yield and patterns of follow-up for BI-RADS category 3 after screening mammography recall in the national mammography database","volume":"296","author":"Berg","year":"2020","journal-title":"Radiology"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12871-021-01245-5","article-title":"Cut-off points between pain intensities of the postoperative pain using receiver operating characteristic (ROC) curves","volume":"21","author":"Cho","year":"2021","journal-title":"BMC Anesthesiol."},{"key":"B6","doi-asserted-by":"publisher","first-page":"e468","DOI":"10.1016\/S2589-7500(20)30185-0","article-title":"Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study","volume":"2","author":"Dembrower","year":"2020","journal-title":"Lancet Digital Health"},{"key":"B7","doi-asserted-by":"publisher","first-page":"860","DOI":"10.2214\/AJR.20.24894","article-title":"Current status and future of BI-RADS in multimodality imaging, from the AJR special series on radiology reporting and data systems","volume":"216","author":"Eghtedari","year":"2021","journal-title":"Am. J. Roentgenol."},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-023-00753-7","article-title":"Making machine learning matter to clinicians: model actionability in medical decision-making","volume":"6","author":"Ehrmann","year":"2023","journal-title":"npj Digit. Med"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0064366","article-title":"If you don't find it often, you often don't find it: why some cancers are missed in breast cancer screening","volume":"8","author":"Evans","year":"2013","journal-title":"PLoS ONE"},{"key":"B10","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1148\/radiol.2019182627","article-title":"Artificial intelligence for mammography and digital breast tomosynthesis: Current concepts and future perspectives","volume":"293","author":"Geras","year":"2019","journal-title":"Radiology"},{"key":"B11","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/0022-1759(95)00121-P","article-title":"A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests","volume":"185","author":"Greiner","year":"1995","journal-title":"J. Immunol. Met."},{"key":"B12","doi-asserted-by":"publisher","first-page":"297","DOI":"10.11613\/BM.2016.034","article-title":"On determining the most appropriate test cut-off value: The case of tests with continuous results","volume":"26","author":"Habibzadeh","year":"2016","journal-title":"Biochemia Medica"},{"key":"B13","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1148\/radiol.2021210391","article-title":"Machine learning for workflow applications in screening mammography: systematic review and meta-analysis","volume":"302","author":"Hickman","year":"2022","journal-title":"Radiology"},{"key":"B14","doi-asserted-by":"publisher","first-page":"5979","DOI":"10.1038\/s41598-022-09954-8","article-title":"On evaluation metrics for medical applications of artificial intelligence","volume":"12","author":"Hicks","year":"2022","journal-title":"Sci. Rep."},{"key":"B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ejrad.2023.110913","article-title":"Breast cancer missed at screening; hindsight or mistakes?","volume":"165","author":"Hovda","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"B16","doi-asserted-by":"publisher","first-page":"5748","DOI":"10.18632\/aging.202502","article-title":"Global incidence and mortality of breast cancer: a trend analysis","volume":"13","author":"Huang","year":"2021","journal-title":"Aging"},{"key":"B17","doi-asserted-by":"publisher","first-page":"36","DOI":"10.17116\/profmed20242701136","article-title":"Socio-economic efficiency of the early cancer detection during the medical checkup","volume":"27","author":"Ignatyeva","year":"2024","journal-title":"Russian J. Prevent. Med."},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.47511","article-title":"Age-Related Incidence and Peak Occurrence of Contralateral Breast Cancer","author":"Kim","year":"2023","journal-title":"JAMA Net. Open"},{"key":"B19","doi-asserted-by":"publisher","first-page":"e138","DOI":"10.1016\/S2589-7500(20)30003-0","article-title":"Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study","volume":"2","author":"Kim","year":"2020","journal-title":"Lancet Digital Health"},{"key":"B20","doi-asserted-by":"publisher","first-page":"32","DOI":"10.17116\/profmed20242705132","article-title":"The impact of artificial intelligence on double reading of mammograms","volume":"25","author":"Kudryavtsev","year":"2024","journal-title":"Profilakticheskaya meditsina"},{"key":"B21","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1007\/s13312-011-0055-4","article-title":"Receiver operating characteristic (ROC). curve for medical researchers","volume":"48","author":"Kumar","year":"2011","journal-title":"Indian Pediat."},{"key":"B22","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/j.crad.2019.02.006","article-title":"Artificial intelligence in breast imaging","volume":"74","author":"Le","year":"2019","journal-title":"Clin. Radiol."},{"key":"B23","doi-asserted-by":"publisher","first-page":"2341","DOI":"10.1007\/s10238-022-00895-0","article-title":"Mammography diagnosis of breast cancer screening through machine learning: a systematic review and meta-analysis","volume":"23","author":"Liu","year":"2022","journal-title":"Clin. Exp. Med."},{"key":"B24","doi-asserted-by":"publisher","first-page":"1105","DOI":"10.1007\/s11596-021-2474-3","article-title":"Application of artificial intelligence in medicine: an overview","volume":"41","author":"Liu","year":"2021","journal-title":"Curr. Med. Sci."},{"key":"B25","doi-asserted-by":"publisher","first-page":"1806","DOI":"10.1001\/jama.2019.16489","article-title":"How to read articles that use machine learning","volume":"322","author":"Liu","year":"2019","journal-title":"JAMA"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1515\/ijb-2018-0060","article-title":"The youden index in the generalized receiver operating characteristic curve context","volume":"15","author":"Mart\u00ednez-Camblor","year":"2019","journal-title":"Int. J. Biostatist."},{"key":"B27","author":"Morozov","year":""},{"key":"B28","volume-title":"Organization of a population screening program for the breast cancer among the female population: Methodological recommendations","author":"Morozov","year":""},{"key":"B29","first-page":"629","article-title":"Moscow screening: breast cancer screening with mammography as a method of improving early stage cancer detection","volume":"27","author":"Morozov","year":"2019","journal-title":"Problemy sotsial'noi gigieny, zdravookhraneniia i istorii meditsiny"},{"key":"B30","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1080\/17453674.2021.1918389","article-title":"Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal","volume":"92","author":"Olczak","year":"2021","journal-title":"Acta Orthopaedica"},{"key":"B31","doi-asserted-by":"publisher","first-page":"603","DOI":"10.37469\/0507-3758-2020-66-6-603-608","article-title":"Prospect of application of artificial intelligence systems for breast cancer screening","volume":"66","author":"Ovsyannikov","year":"2020","journal-title":"Probl. Oncol."},{"key":"B32","doi-asserted-by":"publisher","first-page":"1504","DOI":"10.1001\/jamaoncol.2018.1901","article-title":"Cost-effectiveness and benefit-to-harm ratio of risk-stratified screening for breast cancer a life-table model","volume":"4","author":"Pashayan","year":"2018","journal-title":"JAMA Oncol."},{"key":"B33","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1007\/s40273-020-00914-6","article-title":"Calculating and interpreting ICERs and net benefit","volume":"38","author":"Paulden","year":"2020","journal-title":"PharmacoEconomics"},{"key":"B34","doi-asserted-by":"publisher","first-page":"22","DOI":"10.17816\/DD83184","article-title":"Comparative study of the digital mammography data analysis system based on artificial intelligence \u2018Celsus' and radiologists","volume":"2","author":"Pavlovich","year":"2021","journal-title":"Digital Diagnost."},{"key":"B35","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1126\/science.ns-4.93.453-a","article-title":"The numerical measure of the success of predictions","volume":"4","author":"Peirce","year":"1884","journal-title":"Science"},{"key":"B36","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1002\/sim.2929","article-title":"Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond","volume":"27","author":"Pencina","year":"2008","journal-title":"Statist. Med."},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533166","article-title":"Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare","author":"Pfohl","year":"2022","journal-title":"2022 ACM Conference on Fairness, Accountability, and Transparency. New York, NY, USA: ACM"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.2165\/00019053-200321060-00003","article-title":"Breast cancer management: quality-of-life and cost considerations","author":"Radice","year":"2003","journal-title":"Pharmacoeconomics"},{"key":"B39","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1186\/1472-6947-11-45","article-title":"Decision curve analysis revisited: overall net benefit, relationships to ROC curve analysis, and application to case-control studies","volume":"11","author":"Rousson","year":"2011","journal-title":"BMC Med. Informat. Dec. Making"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1581","DOI":"10.1001\/jamaoncol.2020.3321","article-title":"External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms","volume":"6","author":"Salim","year":"2020","journal-title":"JAMA Oncol."},{"key":"B41","doi-asserted-by":"publisher","first-page":"115","DOI":"10.3747\/co.25.3770","article-title":"Screening for breast cancer in 2018\u2014what should we be doing today?","volume":"25","author":"Seely","year":"2018","journal-title":"Curr. Oncol."},{"key":"B42","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1093\/jamia\/ocad006","article-title":"Assessing the net benefit of machine learning models in the presence of resource constraints","volume":"30","author":"Singh","year":"2023","journal-title":"JAMIA"},{"key":"B43","doi-asserted-by":"publisher","first-page":"20190610","DOI":"10.1259\/bjr.20190610","article-title":"Double reading in breast cancer screening: considerations for policy-making","volume":"93","author":"Taylor-Phillips","year":"2020","journal-title":"Br. J. Radiol"},{"key":"B44","doi-asserted-by":"publisher","first-page":"822","DOI":"10.3390\/diagnostics15070822","article-title":"Evolution of an artificial intelligence-powered application for mammography","volume":"15","author":"Vasilev","year":"2025","journal-title":"Diagnostics"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.21045\/1811-0185-2023-8-54-67","article-title":"The first 10,000 mammography exams performed as part of the \u2018Description and interpretation of mammography data using artificial intelligence' service","author":"Vasilev","year":"","journal-title":"Manager Zdravookhranenia"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.17116\/profmed20242707123","article-title":"Autonomous artificial intelligence for sorting the preventive imaging studies' results","author":"Vasilev","year":"2024","journal-title":"Russian J. Prevent. Med."},{"key":"B47","doi-asserted-by":"publisher","first-page":"105190","DOI":"10.1016\/j.ijmedinf.2023.105190","article-title":"Clinical application of radiological AI for pulmonary nodule evaluation: Replicability and susceptibility to the population shift caused by the COVID-19 pandemic","volume":"178","author":"Vasilev","year":"","journal-title":"Int. J. Med. Informat."},{"key":"B48","doi-asserted-by":"publisher","first-page":"93","DOI":"10.17816\/DD321423","article-title":"Double-reading mammograms using artificial intelligence technologies: A new model of mass preventive examination organization","volume":"4","author":"Vasilev","year":"","journal-title":"Digital Diagnost."},{"key":"B49","doi-asserted-by":"publisher","first-page":"i6","DOI":"10.1136\/bmj.i6","article-title":"Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests","volume":"352","author":"Vickers","year":"2016","journal-title":"BMJ"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s41512-019-0064-7","article-title":"A simple, step-by-step guide to interpreting decision curve analysis","volume":"3","author":"Vickers","year":"2019","journal-title":"Diagnost. Prognost. Res."},{"key":"B51","doi-asserted-by":"publisher","first-page":"1658","DOI":"10.34172\/ijhpm.2021.62","article-title":"Assessment of the benefits and cost-effectiveness of population-based breast cancer screening in urban China: a model-based analysis","volume":"11","author":"Wang","year":"2022","journal-title":"Int. J. Health Policy Manag."},{"key":"B52","year":"2022","journal-title":"A short guide to cancer screening."},{"key":"B53","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1002\/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3","article-title":"Index for rating diagnostic tests","volume":"3","author":"Youden","year":"1950","journal-title":"Cancer"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2025.1690955\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:15:41Z","timestamp":1763442941000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2025.1690955\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,18]]},"references-count":53,"alternative-id":["10.3389\/fdata.2025.1690955"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2025.1690955","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,18]]},"article-number":"1690955"}}