{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T04:31:37Z","timestamp":1747801897706,"version":"3.41.0"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,10,15]],"date-time":"2024-10-15T00:00:00Z","timestamp":1728950400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,15]],"date-time":"2024-10-15T00:00:00Z","timestamp":1728950400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001171","name":"Cancer Institute NSW","doi-asserted-by":"publisher","award":["2022\/ECF1426"],"award-info":[{"award-number":["2022\/ECF1426"]}],"id":[{"id":"10.13039\/501100001171","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000925","name":"National Health and Medical Research Council","doi-asserted-by":"publisher","award":["APP1162872"],"award-info":[{"award-number":["APP1162872"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001026","name":"National Breast Cancer Foundation","doi-asserted-by":"publisher","award":["IIRS-18-089"],"award-info":[{"award-number":["IIRS-18-089"]}],"id":[{"id":"10.13039\/501100001026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001774","name":"University of Sydney","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001774","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging. Inform. med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>This study aims to investigate whether global mammographic radiomic features (GMRFs) can distinguish <jats:italic>hardest-<\/jats:italic> from <jats:italic>easiest-to-interpret<\/jats:italic> normal cases for radiology trainees (RTs). Data from 137 RTs were analysed, with each interpreting seven educational self-assessment test sets comprising 60 cases (40 normal and 20 cancer). The study only examined normal cases. Difficulty scores were computed based on the percentage of readers who incorrectly classified each case, leading to their classification as <jats:italic>hardest-<\/jats:italic> or <jats:italic>easiest-to-interpret<\/jats:italic> based on whether their difficulty scores fell within and above the 75th or within and below the 25th percentile, respectively (resulted in 140 cases in total used). Fifty-nine <jats:italic>low-density<\/jats:italic> and 81 <jats:italic>high-density<\/jats:italic> cases were identified. Thirty-four GMRFs were extracted for each case. A random forest machine learning model was trained to differentiate between <jats:italic>hardest-<\/jats:italic> and <jats:italic>easiest-to-interpret<\/jats:italic> normal cases and validated using leave-one-out-cross-validation approach. The model\u2019s performance was evaluated using the area under receiver operating characteristic curve (AUC). Significant features were identified through feature importance analysis. Difference between <jats:italic>hardest-<\/jats:italic> and <jats:italic>easiest-to-interpret<\/jats:italic> cases among 34 GMRFs and in difficulty level between <jats:italic>low-<\/jats:italic> and <jats:italic>high-density<\/jats:italic> cases was tested using Kruskal\u2013Wallis. The model achieved AUC\u2009=\u20090.75 with <jats:italic>cluster prominence<\/jats:italic> and <jats:italic>range<\/jats:italic> emerging as the most useful features. Fifteen GMRFs differed significantly (<jats:italic>p<\/jats:italic>\u2009&lt;\u20090.05) between <jats:italic>hardest-<\/jats:italic> and <jats:italic>easiest-to-interpret<\/jats:italic> cases. Difficulty level among <jats:italic>low-<\/jats:italic> vs <jats:italic>high-density<\/jats:italic> cases did not differ significantly (<jats:italic>p<\/jats:italic>\u2009=\u20090.12). GMRFs can predict <jats:italic>hardest-to-interpret<\/jats:italic> normal cases for RTs, underscoring the importance of GMRFs in identifying the most difficult normal cases for RTs and facilitating customised training programmes tailored to trainees\u2019 learning needs.<\/jats:p>","DOI":"10.1007\/s10278-024-01291-8","type":"journal-article","created":{"date-parts":[[2024,10,15]],"date-time":"2024-10-15T17:02:04Z","timestamp":1729011724000},"page":"1904-1913","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Machine Learning Model Based on Global Mammographic Radiomic Features Can Predict Which Normal Mammographic Cases Radiology Trainees Find Most Difficult"],"prefix":"10.1007","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2891-9217","authenticated-orcid":false,"given":"Somphone","family":"Siviengphanom","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patrick C.","family":"Brennan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah J.","family":"Lewis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Phuong Dung","family":"Trieu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziba","family":"Gandomkar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,10,15]]},"reference":[{"key":"1291_CR1","doi-asserted-by":"crossref","unstructured":"Sung H, et al.: Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA: A Cancer Journal for Clinicians 71:209\u2013249, 2021","DOI":"10.3322\/caac.21660"},{"key":"1291_CR2","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1258\/jms.2012.012077","volume":"19","author":"E Paci","year":"2012","unstructured":"Paci E: Summary of the Evidence of Breast Cancer Service Screening Outcomes in Europe and First Estimate of the Benefit and Harm Balance Sheet. Journal of Medical Screening 19:5-13, 2012","journal-title":"Journal of Medical Screening"},{"key":"1291_CR3","first-page":"291","volume":"19","author":"EU Ekpo","year":"2018","unstructured":"Ekpo EU, Alakhras M, Brennan P: Errors in Mammography Cannot be Solved Through Technology Alone. Asian Pac J Cancer Prev 19:291-301, 2018","journal-title":"Asian Pac J Cancer Prev"},{"key":"1291_CR4","doi-asserted-by":"publisher","first-page":"091907","DOI":"10.1118\/1.4892173","volume":"41","author":"J Zhang","year":"2014","unstructured":"Zhang J, Lo JY, Kuzmiak CM, Ghate SV, Yoon SC, Mazurowski MA: Using computer-extracted image features for modeling of error-making patterns in detection of mammographic masses among radiology residents. Medical Physics 41:091907, 2014","journal-title":"Medical Physics"},{"key":"1291_CR5","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1016\/j.jbi.2015.01.007","volume":"54","author":"J Zhang","year":"2015","unstructured":"Zhang J, Silber JI, Mazurowski MA: Modeling false positive error making patterns in radiology trainees for improved mammography education. Journal of Biomedical Informatics 54:50-57, 2015","journal-title":"Journal of Biomedical Informatics"},{"key":"1291_CR6","doi-asserted-by":"crossref","unstructured":"Wong DJ, et al.: Do reader characteristics affect diagnostic efficacy in screening mammography? A systematic review. Clinical Breast Cancer, 2023","DOI":"10.1016\/j.clbc.2023.01.009"},{"key":"1291_CR7","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1007\/s12282-022-01335-3","volume":"29","author":"Z Gandomkar","year":"2022","unstructured":"Gandomkar Z, Lewis SJ, Li T, Ekpo EU, Brennan PC: A machine learning model based on readers\u2019 characteristics to predict their performances in reading screening mammograms. Breast Cancer 29:589-598, 2022","journal-title":"Breast Cancer"},{"key":"1291_CR8","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1177\/0969141315610790","volume":"23","author":"S Hofvind","year":"2016","unstructured":"Hofvind S, et al.: Audit feedback on reading performance of screening mammograms: An international comparison. Journal of Medical Screening 23:150-159, 2016","journal-title":"Journal of Medical Screening"},{"key":"1291_CR9","doi-asserted-by":"publisher","first-page":"S49","DOI":"10.1016\/j.radi.2020.06.006","volume":"26","author":"ME Suleiman","year":"2020","unstructured":"Suleiman ME, Rickard M, Brennan PC: Perfecting detection through education. Radiography 26:S49-S53, 2020","journal-title":"Radiography"},{"key":"1291_CR10","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1002\/jmrs.6","volume":"60","author":"W Lee","year":"2013","unstructured":"Lee W, Peters G: Mammographic screening for breast cancer: A review. Journal of Medical Radiation Sciences 60:35-39, 2013","journal-title":"Journal of Medical Radiation Sciences"},{"key":"1291_CR11","doi-asserted-by":"publisher","first-page":"1541","DOI":"10.1007\/s10278-023-00836-7","volume":"36","author":"S Siviengphanom","year":"2023","unstructured":"Siviengphanom S, Gandomkar Z, Lewis SJ, Brennan PC: Global radiomic features from mammography for predicting difficult-to-interpret normal cases. Journal of Digital Imaging 36:1541\u20131552, 2023","journal-title":"Journal of Digital Imaging"},{"key":"1291_CR12","doi-asserted-by":"publisher","first-page":"157","DOI":"10.2147\/CEOR.S150260","volume":"10","author":"A Vlahiotis","year":"2018","unstructured":"Vlahiotis A, Griffin B, Stavros Md FAT, Margolis J: Analysis of utilization patterns and associated costs of the breast imaging and diagnostic procedures after screening mammography. ClinicoEconomics and Outcomes Research Volume 10:157-167, 2018","journal-title":"ClinicoEconomics and Outcomes Research"},{"key":"1291_CR13","doi-asserted-by":"publisher","first-page":"20190057","DOI":"10.1259\/bjr.20190057","volume":"92","author":"Z Gandomkar","year":"2019","unstructured":"Gandomkar Z, Mello-Thoms C: Visual search in breast imaging. Br J Radiol 92:20190057, 2019","journal-title":"Br J Radiol"},{"key":"1291_CR14","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1016\/S1076-6332(99)80252-9","volume":"6","author":"CF Nodine","year":"1999","unstructured":"Nodine CF, et al.: How experience and training influence mammography expertise. Acad Radiol 6:575-585, 1999","journal-title":"Acad Radiol"},{"key":"1291_CR15","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1148\/radiol.10101698","volume":"259","author":"DS Buist","year":"2011","unstructured":"Buist DS, et al.: Influence of annual interpretive volume on screening mammography performance in the United States. Radiology 259:72-84, 2011","journal-title":"Radiology"},{"key":"1291_CR16","doi-asserted-by":"crossref","unstructured":"Siviengphanom S, Gandomkar Z, Lewis SJ, Brennan PC: Global mammographic radiomic signature can predict radiologists\u2019 difficult-to-interpret normal cases. Proc. SPIE Medical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment: City","DOI":"10.1117\/12.2645377"},{"key":"1291_CR17","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1111\/1754-9485.12576","volume":"61","author":"ZZ Ang","year":"2017","unstructured":"Ang ZZ, Rawashdeh MA, Heard R, Brennan PC, Lee W, Lewis SJ: Classification of normal screening mammograms is strongly influenced by perceived mammographic breast density. Journal of Medical Imaging and Radiation Oncology 61:461-469, 2017","journal-title":"Journal of Medical Imaging and Radiation Oncology"},{"key":"1291_CR18","unstructured":"Darker I, Chen Y, Gale A: Health professionals' agreement on density judgements and successful abnormality identification within the UK Breast Screening Programme. Proc. SPIE Medical Imaging: City"},{"key":"1291_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0284605","volume":"18","author":"Z Gandomkar","year":"2023","unstructured":"Gandomkar Z, et al.: Reliability of radiologists\u2019 first impression when interpreting a screening mammogram. PLOS ONE 18:1-19, 2023","journal-title":"PLOS ONE"},{"key":"1291_CR20","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1093\/bjr\/tqad025","volume":"97","author":"S Siviengphanom","year":"2024","unstructured":"Siviengphanom S, Lewis SJ, Brennan PC, Gandomkar Z: Computer-extracted global radiomic features can predict the radiologists\u2019 first impression about the abnormality of a screening mammogram. British Journal of Radiology 97:168\u2013179, 2024","journal-title":"British Journal of Radiology"},{"key":"1291_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-018-26100-5","volume":"8","author":"PC Brennan","year":"2018","unstructured":"Brennan PC, et al.: Radiologists can detect the 'gist' of breast cancer before any overt signs of cancer appear. Sci Rep 8:1-12, 2018","journal-title":"Sci Rep"},{"key":"1291_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-021-99582-5","volume":"11","author":"Z Gandomkar","year":"2021","unstructured":"Gandomkar Z, et al.: Global processing provides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection. Sci Rep 11:1-12, 2021","journal-title":"Sci Rep"},{"key":"1291_CR23","doi-asserted-by":"crossref","unstructured":"Siviengphanom S, Lewis SJ, Brennan PC, Gandomkar Z: Predicting the gist of breast cancer on a screening mammogram using global radiomic features. Proc. SPIE Medical Imaging 2024: Image Perception, Observer Performance, and Technology Assessment: City","DOI":"10.1117\/12.3005470"},{"key":"1291_CR24","doi-asserted-by":"publisher","first-page":"1170","DOI":"10.3758\/s13423-013-0459-3","volume":"20","author":"KK Evans","year":"2013","unstructured":"Evans KK, Georgian-Smith D, Tambouret R, Birdwell RL, Wolfe JM: The gist of the abnormal: Above-chance medical decision making in the blink of an eye. Psychonomic Bulletin & Review 20:1170-1175, 2013","journal-title":"Psychonomic Bulletin & Review"},{"key":"1291_CR25","first-page":"563","volume":"278","author":"RJ Gillies","year":"2016","unstructured":"Gillies RJ, Kinahan PE, Hricak H: Radiomics: Images Are More than Pictures, They Are Data. Radiology 278:563-577, 2016","journal-title":"They Are Data. Radiology"},{"key":"1291_CR26","doi-asserted-by":"publisher","first-page":"1228","DOI":"10.1016\/j.acra.2021.09.025","volume":"29","author":"S Siviengphanom","year":"2021","unstructured":"Siviengphanom S, Gandomkar Z, Lewis SJ, Brennan PC: Mammography-based Radiomics in Breast Cancer: A Scoping Review of Current Knowledge and Future Needs. Academic Radiology 29:1228-1247, 2021","journal-title":"Academic Radiology"},{"key":"1291_CR27","doi-asserted-by":"crossref","unstructured":"Gandomkar Z, et al.: An end-to-end deep learning model can detect the gist of the abnormal in prior mammograms as perceived by experienced radiologists. Proc. SPIE Medical Imaging 2021: Image Perception, Observer Performance, and Technology Assessment: City","DOI":"10.1117\/12.2582099"},{"key":"1291_CR28","volume-title":"Breast Screen Reader Assessment Strategy (BREAST): A Research Infrastructure with a Translational Objective","author":"P Brennan","year":"2018","unstructured":"Brennan P, Lee W, Tapia K: Breast Screen Reader Assessment Strategy (BREAST): A Research Infrastructure with a Translational Objective, Cambridge: Cambridge University Press, 2018"},{"key":"1291_CR29","doi-asserted-by":"crossref","unstructured":"McEntee M, Brennan P, Evanoff M, Phillps P, O Connor W, Manning D: Optimum ambient lighting conditions for the viewing of softcopy radiological images: SPIE, 2006","DOI":"10.1117\/12.660137"},{"key":"1291_CR30","doi-asserted-by":"publisher","first-page":"4149","DOI":"10.1118\/1.4921996","volume":"42","author":"Y Zheng","year":"2015","unstructured":"Zheng Y, et al.: Parenchymal texture analysis in digital mammography: A fully automated pipeline for breast cancer risk assessment. Medical Physics 42:4149-4160, 2015","journal-title":"Medical Physics"},{"key":"1291_CR31","doi-asserted-by":"crossref","unstructured":"Gandomkar Z, Suleiman M, Demchig D, Brennan P, McEntee M: BI-RADS density categorization using deep neural networks. Proc. SPIE 10952, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment: City","DOI":"10.1117\/12.2513185"},{"key":"1291_CR32","doi-asserted-by":"publisher","first-page":"35","DOI":"10.2152\/jmi.66.35","volume":"66","author":"A Haga","year":"2019","unstructured":"Haga A, et al.: Standardization of imaging features for radiomics analysis. The Journal of Medical Investigation 66:35-37, 2019","journal-title":"The Journal of Medical Investigation"},{"key":"1291_CR33","doi-asserted-by":"crossref","unstructured":"Haralick RM, Shanmugam K, Dinstein IH: Textural Features for Image Classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-3:610\u2013621, 1973","DOI":"10.1109\/TSMC.1973.4309314"},{"key":"1291_CR34","doi-asserted-by":"publisher","first-page":"1264","DOI":"10.1109\/21.44046","volume":"19","author":"M Amadasun","year":"1989","unstructured":"Amadasun M, King R: Textural features corresponding to textural properties. IEEE Transactions on Systems, Man, and Cybernetics 19:1264-1274, 1989","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"1291_CR35","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L: Random Forests. Machine Learning 45:5-32, 2001","journal-title":"Machine Learning"},{"key":"1291_CR36","doi-asserted-by":"publisher","first-page":"3301","DOI":"10.1093\/bioinformatics\/bti499","volume":"21","author":"AM Molinaro","year":"2005","unstructured":"Molinaro AM, Simon R, Pfeiffer RM: Prediction error estimation: a comparison of resampling methods. Bioinformatics 21:3301-3307, 2005","journal-title":"Bioinformatics"},{"key":"1291_CR37","doi-asserted-by":"crossref","unstructured":"Ledesma RD, Valero-Mora P, Macbeth G: The Scree Test and the Number of Factors: a Dynamic Graphics Approach. The Spanish Journal of Psychology 18, 2015","DOI":"10.1017\/sjp.2015.13"},{"key":"1291_CR38","doi-asserted-by":"crossref","unstructured":"A Tapia K, Rickard MT, McEntee MF, Garvey G, Lydiard L, C Brennan P: Impact of breast density on cancer detection: observations from digital mammography test sets. International Journal of Radiology & Radiation Therapy 7:36\u201341, 2020","DOI":"10.15406\/ijrrt.2020.07.00261"},{"key":"1291_CR39","doi-asserted-by":"publisher","first-page":"888","DOI":"10.1016\/j.acra.2014.01.025","volume":"21","author":"LJ Grimm","year":"2014","unstructured":"Grimm LJ, Kuzmiak CM, Ghate SV, Yoon SC, Mazurowski MA: Radiology Resident Mammography Training. Academic Radiology 21:888-892, 2014","journal-title":"Academic Radiology"},{"key":"1291_CR40","doi-asserted-by":"crossref","unstructured":"Pertuz S, Torres GF, Tamimi R, Kamarainen J: Open Framework for Mammography-based Breast Cancer Risk Assessment. Proc. 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI): City","DOI":"10.1109\/BHI.2019.8834599"}],"container-title":["Journal of Imaging Informatics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-024-01291-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-024-01291-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-024-01291-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T17:27:19Z","timestamp":1747762039000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-024-01291-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,15]]},"references-count":40,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["1291"],"URL":"https:\/\/doi.org\/10.1007\/s10278-024-01291-8","relation":{},"ISSN":["2948-2933"],"issn-type":[{"type":"electronic","value":"2948-2933"}],"subject":[],"published":{"date-parts":[[2024,10,15]]},"assertion":[{"value":"1 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 September 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 September 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 October 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ethical approval was obtained for this retrospective study from the Human Research Ethics Committee of the University of Sydney [Protocol number 2019\/013 and 2017\/028].","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"Informed consent was obtained from all individual readers included in the study.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Informed consent was obtained from all individual readers included in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"P.C.B. and Z.G. as well as an academic at the University of Sydney are CEO and Co-founder (P.C.B.)\/employee (Z.G.) of DetectED-X, a company which aims to educate radiologists. The paper will not impact on DetectED-X\u2019s activities. All other authors of this manuscript, other than being employee of Western Sydney University (S.J.L.), employee (P.D.T.)\/student (S.S.) of the University of Sydney, employee of the University of Technology Sydney (S.S.), declare no relationships with any companies, whose products or services may be related to the subject matter of the article.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}