{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T17:54:54Z","timestamp":1789667694018,"version":"4.0.1"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T00:00:00Z","timestamp":1726617600000},"content-version":"vor","delay-in-days":16,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["K08CA279881"],"award-info":[{"award-number":["K08CA279881"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,9,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>Clinical utility data on pulmonary nodule (PN) risk stratification biomarkers are lacking. We aimed to determine the incremental predictive value and clinical utility of using an artificial intelligence (AI) radiomics-based computer-aided diagnosis (CAD) tool in addition to routine clinical information to risk stratify PNs among real-world patients.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>We performed a retrospective cohort study of patients with PNs who underwent lung biopsy. We collected clinical data and used a commercially available AI radiomics-based CAD tool to calculate a Lung Cancer Prediction (LCP) score. We developed logistic regression models to evaluate a well-validated clinical risk prediction model (the Mayo Clinic model) with and without the LCP score (Mayo vs Mayo + LCP) using area under the curve (AUC), risk stratification table, and standardized net benefit analyses.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Among the 134 patients undergoing PN biopsy, cancer prevalence was 61%. Addition of the radiomics-based LCP score to the Mayo model was associated with increased predictive accuracy (likelihood ratio test, P\u2009=\u2009.012). The AUCs for the Mayo and Mayo + LCP models were 0.58 (95% CI = 0.48 to 0.69) and 0.65 (95% CI = 0.56 to 0.75), respectively. At the 65% risk threshold, the Mayo + LCP model was associated with increased sensitivity (56% vs 38%; P\u2009=\u2009.019), similar false positive rate (33% vs 35%; P\u2009=\u2009.8), and increased standardized net benefit (18% vs -3.3%) compared with the Mayo model.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Use of a commercially available AI radiomics-based CAD tool as a supplement to clinical information improved PN cancer risk prediction and may result in clinically meaningful changes in risk stratification.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jncics\/pkae086","type":"journal-article","created":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T16:33:13Z","timestamp":1726677193000},"source":"Crossref","is-referenced-by-count":7,"title":["Clinical utility of an artificial intelligence radiomics-based tool for risk stratification of pulmonary nodules"],"prefix":"10.1093","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8262-484X","authenticated-orcid":false,"given":"Roger Y","family":"Kim","sequence":"first","affiliation":[{"name":"Division of Pulmonary, Allergy and Critical Care, Department of Medicine, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Clarisa","family":"Yee","sequence":"additional","affiliation":[{"name":"NYU Langone Health , New York City, NY,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-5675-5525","authenticated-orcid":false,"given":"Sana","family":"Zeb","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy and Critical Care, Department of Medicine, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer","family":"Steltz","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy and Critical Care, Department of Medicine, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1525-6503","authenticated-orcid":false,"given":"Andrew J","family":"Vickers","sequence":"additional","affiliation":[{"name":"Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center , New York City, NY,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7761-8728","authenticated-orcid":false,"given":"Katharine A","family":"Rendle","sequence":"additional","affiliation":[{"name":"Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7714-3910","authenticated-orcid":false,"given":"Nandita","family":"Mitra","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lyndsey C","family":"Pickup","sequence":"additional","affiliation":[{"name":"Optellum Ltd , Oxford,","place":["UK"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David M","family":"DiBardino","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy and Critical Care, Department of Medicine, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3871-8697","authenticated-orcid":false,"given":"Anil","family":"Vachani","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy and Critical Care, Department of Medicine, University of Pennsylvania , Philadelphia, PA,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,9,18]]},"reference":[{"issue":"10","key":"2024102921591098300_pkae086-B1","doi-asserted-by":"publisher","first-page":"1208","DOI":"10.1164\/rccm.201505-0990OC","article-title":"Recent trends in the identification of incidental pulmonary nodules","volume":"192","author":"Gould","year":"2015","journal-title":"Am J Respir Crit Care Med"},{"issue":"6","key":"2024102921591098300_pkae086-B2","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1513\/AnnalsATS.201609-727OC","article-title":"Clinical equipoise and shared decision-making in pulmonary nodule management: a survey of American Thoracic Society Clinicians","volume":"14","author":"Iaccarino","year":"2017","journal-title":"Ann Am Thorac Soc"},{"issue":"4","key":"2024102921591098300_pkae086-B3","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1164\/rccm.201104-0679CI","article-title":"Decision making in patients with pulmonary nodules","volume":"185","author":"Ost","year":"2012","journal-title":"Am J Respir Crit Care Med"},{"issue":"1","key":"2024102921591098300_pkae086-B4","doi-asserted-by":"publisher","first-page":"18","DOI":"10.7326\/M23-0653","article-title":"Rates of downstream procedures and complications associated with lung cancer screening in routine clinical practice: a retrospective cohort study","volume":"177","author":"Rendle","year":"2024","journal-title":"Ann Intern Med"},{"issue":"suppl 5","key":"2024102921591098300_pkae086-B5","doi-asserted-by":"publisher","first-page":"e93S","DOI":"10.1378\/chest.12-2351","article-title":"Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines","volume":"143","author":"Gould","year":"2013","journal-title":"Chest"},{"issue":"2","key":"2024102921591098300_pkae086-B6","doi-asserted-by":"publisher","first-page":"e89","DOI":"10.1016\/j.cllc.2019.05.004","article-title":"Effect of a rule-in biomarker test on pulmonary nodule management: a survey of pulmonologists and thoracic surgeons","volume":"21","author":"Tanner","year":"2020","journal-title":"Clin Lung Cancer"},{"issue":"10","key":"2024102921591098300_pkae086-B7","doi-asserted-by":"publisher","first-page":"1117","DOI":"10.1513\/AnnalsATS.201803-173CME","article-title":"Models to estimate the probability of malignancy in patients with pulmonary nodules","volume":"15","author":"Choi","year":"2018","journal-title":"Ann Am Thorac Soc"},{"issue":"2","key":"2024102921591098300_pkae086-B8","doi-asserted-by":"publisher","first-page":"562","DOI":"10.1016\/j.chest.2021.07.2168","article-title":"The probability of lung cancer in patients with incidentally detected pulmonary nodules: clinical characteristics and accuracy of prediction models","volume":"161","author":"Vachani","year":"2021","journal-title":"Chest"},{"issue":"8","key":"2024102921591098300_pkae086-B9","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1001\/archinte.1997.00440290031002","article-title":"The probability of malignancy in solitary pulmonary nodules","volume":"157","author":"Swensen","year":"1997","journal-title":"Arch Intern Med"},{"issue":"4","key":"2024102921591098300_pkae086-B10","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1136\/thx.2007.084731","article-title":"Validation of two models to estimate the probability of malignancy in patients with solitary pulmonary nodules","volume":"63","author":"Schultz","year":"2008","journal-title":"Thorax"},{"issue":"2","key":"2024102921591098300_pkae086-B11","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1016\/j.chest.2017.01.018","article-title":"Physician assessment of pretest probability of malignancy and adherence with guidelines for pulmonary nodule evaluation","volume":"152","author":"Tanner","year":"2017","journal-title":"Chest"},{"issue":"4","key":"2024102921591098300_pkae086-B12","doi-asserted-by":"publisher","first-page":"1028","DOI":"10.1016\/j.chest.2023.05.025","article-title":"Update on biomarkers for the stratification of indeterminate pulmonary nodules","volume":"164","author":"Paez","year":"2023","journal-title":"Chest"},{"issue":"7","key":"2024102921591098300_pkae086-B13","doi-asserted-by":"publisher","first-page":"e15","DOI":"10.1164\/rccm.201708-1678ST","article-title":"Evaluating molecular biomarkers for the early detection of lung cancer: when is a biomarker ready for clinical use? An official American Thoracic Society policy statement","volume":"196","author":"Mazzone","year":"2017","journal-title":"Am J Respir Crit Care Med"},{"issue":"1","key":"2024102921591098300_pkae086-B14","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.crad.2019.04.017","article-title":"Artificial intelligence and radiomics in pulmonary nodule management: current status and future applications","volume":"75","author":"Ather","year":"2020","journal-title":"Clin Radiol"},{"key":"2024102921591098300_pkae086-B15","doi-asserted-by":"publisher","DOI":"10.3233\/CBM-230360","article-title":"Radiomics and artificial intelligence for risk stratification of pulmonary nodules: ready for primetime?","author":"Kim","year":"2024","journal-title":"Cancer Biomark"},{"issue":"2","key":"2024102921591098300_pkae086-B16","doi-asserted-by":"publisher","first-page":"563","DOI":"10.1148\/radiol.2015151169","article-title":"Radiomics: images are more than pictures, they are data","volume":"278","author":"Gillies","year":"2016","journal-title":"Radiology"},{"issue":"6","key":"2024102921591098300_pkae086-B17","doi-asserted-by":"publisher","first-page":"582","DOI":"10.1111\/resp.14502","article-title":"Effect of an artificial intelligence tool on management decisions for indeterminate pulmonary nodules","volume":"28","author":"Kim","year":"2023","journal-title":"Respirology"},{"issue":"3","key":"2024102921591098300_pkae086-B18","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1148\/radiol.212182","article-title":"Artificial intelligence tool for assessment of indeterminate pulmonary nodules detected with CT","volume":"304","author":"Kim","year":"2022","journal-title":"Radiology"},{"issue":"6","key":"2024102921591098300_pkae086-B19","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1513\/AnnalsATS.201305-107OC","article-title":"Accuracy of clinicians and models for estimating the probability that a pulmonary nodule is malignant","volume":"10","author":"Balekian","year":"2013","journal-title":"Ann Am Thorac Soc"},{"issue":"2","key":"2024102921591098300_pkae086-B20","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1164\/rccm.201903-0505OC","article-title":"Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules","volume":"202","author":"Massion","year":"2020","journal-title":"Am J Respir Crit Care Med"},{"issue":"4","key":"2024102921591098300_pkae086-B21","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1136\/thoraxjnl-2019-214104","article-title":"External validation of a convolutional neural network artificial intelligence tool to predict malignancy in pulmonary nodules","volume":"75","author":"Baldwin","year":"2020","journal-title":"Thorax"},{"key":"2024102921591098300_pkae086-B22","author":"Optellum","year":"2024"},{"key":"2024102921591098300_pkae086-B23","author":"Lassiter"},{"key":"2024102921591098300_pkae086-B24","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/1471-2288-11-13","article-title":"One statistical test is sufficient for assessing new predictive markers","volume":"11","author":"Vickers","year":"2011","journal-title":"BMC Med Res Methodol"},{"issue":"23","key":"2024102921591098300_pkae086-B25","doi-asserted-by":"publisher","first-page":"2577","DOI":"10.1002\/sim.5328","article-title":"Misuse of DeLong test to compare AUCs for nested models","volume":"31","author":"Demler","year":"2012","journal-title":"Stat Med"},{"issue":"4","key":"2024102921591098300_pkae086-B26","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1002\/(sici)1097-0258(19960229)15:4&lt;361::Aid-sim168&gt;3.0.Co;2-4","article-title":"Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors","volume":"15","author":"Harrell","year":"1996","journal-title":"Statist Med"},{"key":"2024102921591098300_pkae086-B27","doi-asserted-by":"publisher","first-page":"b606","DOI":"10.1136\/bmj.b606","article-title":"Prognosis and prognostic research: application and impact of prognostic models in clinical practice","volume":"338","author":"Moons","year":"2009","journal-title":"BMJ"},{"issue":"14","key":"2024102921591098300_pkae086-B28","doi-asserted-by":"publisher","first-page":"2390","DOI":"10.1002\/sim.6100","article-title":"A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes","volume":"33","author":"Nattino","year":"2014","journal-title":"Stat Med"},{"issue":"5","key":"2024102921591098300_pkae086-B29","doi-asserted-by":"publisher","first-page":"709","DOI":"10.1002\/sim.6744","article-title":"A new test and graphical tool to assess the goodness of fit of logistic regression models","volume":"35","author":"Nattino","year":"2016","journal-title":"Stat Med"},{"issue":"10","key":"2024102921591098300_pkae086-B30","doi-asserted-by":"publisher","first-page":"751","DOI":"10.7326\/0003-4819-149-10-200811180-00009","article-title":"Assessing the value of risk predictions by using risk stratification tables","volume":"149","author":"Janes","year":"2008","journal-title":"Ann Intern Med"},{"issue":"3","key":"2024102921591098300_pkae086-B31","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1093\/aje\/kwm305","article-title":"Integrating the predictiveness of a marker with its performance as a classifier","volume":"167","author":"Pepe","year":"2008","journal-title":"Am J Epidemiol"},{"issue":"6","key":"2024102921591098300_pkae086-B32","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: a novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med Decis Making"},{"issue":"21","key":"2024102921591098300_pkae086-B33","doi-asserted-by":"publisher","first-page":"2534","DOI":"10.1200\/JClinOncol.2015.65.5654","article-title":"Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use","volume":"34","author":"Kerr","year":"2016","journal-title":"J Clin Oncol"},{"key":"2024102921591098300_pkae086-B34","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1186\/s41512-019-0064-7","article-title":"A simple, step-by-step guide to interpreting decision curve analysis","volume":"3","author":"Vickers","year":"2019","journal-title":"Diagn Progn Res"},{"issue":"10","key":"2024102921591098300_pkae086-B35","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1136\/thoraxjnl-2020-215107","article-title":"Lung cancer probability and clinical outcomes of baseline and new subsolid nodules detected on low-dose CT screening","volume":"76","author":"Kim","year":"2021","journal-title":"Thorax"},{"issue":"8","key":"2024102921591098300_pkae086-B36","doi-asserted-by":"publisher","first-page":"794","DOI":"10.1136\/thoraxjnl-2015-207221","article-title":"The British Thoracic Society guidelines on the investigation and management of pulmonary nodules","volume":"70","author":"Baldwin","year":"2015","journal-title":"Thorax"},{"issue":"1S","key":"2024102921591098300_pkae086-B37","doi-asserted-by":"publisher","first-page":"S3","DOI":"10.1016\/j.chest.2019.10.064","article-title":"Statistical analysis and reporting guidelines for CHEST","volume":"158","author":"Kattan","year":"2020","journal-title":"Chest"},{"key":"2024102921591098300_pkae086-B38","author":"Swensen"},{"key":"2024102921591098300_pkae086-B39","author":"Kim"}],"container-title":["JNCI Cancer Spectrum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jncics\/advance-article-pdf\/doi\/10.1093\/jncics\/pkae086\/59183298\/pkae086.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jncics\/article-pdf\/8\/5\/pkae086\/60212810\/pkae086.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jncics\/article-pdf\/8\/5\/pkae086\/60212810\/pkae086.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,29]],"date-time":"2024-10-29T21:59:21Z","timestamp":1730239161000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jncics\/article\/doi\/10.1093\/jncics\/pkae086\/7760215"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,2]]},"references-count":39,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,9,2]]}},"URL":"https:\/\/doi.org\/10.1093\/jncics\/pkae086","relation":{},"ISSN":["2515-5091"],"issn-type":[{"value":"2515-5091","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,10]]},"published":{"date-parts":[[2024,9,2]]},"article-number":"pkae086"}}