{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T15:45:53Z","timestamp":1790696753888,"version":"4.1.0"},"reference-count":39,"publisher":"Georg Thieme Verlag KG","issue":"04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\n          Background\u2003Incidental radiographic findings, such as adrenal nodules, are commonly identified in imaging studies and documented in radiology reports. However, patients with such findings frequently do not receive appropriate follow-up, partially due to the lack of tools for the management of such findings and the time required to maintain up-to-date lists. Natural language processing (NLP) is capable of extracting information from free-text clinical documents and could provide the basis for software solutions that do not require changes to clinical workflows.<\/jats:p><jats:p>\n          Objectives\u2003In this manuscript we present (1) a machine learning algorithm we trained to identify radiology reports documenting the presence of a newly discovered adrenal incidentaloma, and (2) the web application and results database we developed to manage these clinical findings.<\/jats:p><jats:p>\n          Methods\u2003We manually annotated a training corpus of 4,090 radiology reports from across our institution with a binary label indicating whether or not a report contains a newly discovered adrenal incidentaloma. We trained a convolutional neural network to perform this text classification task. Over the NLP backbone we built a web application that allows users to coordinate clinical management of adrenal incidentalomas in real time.<\/jats:p><jats:p>\n          Results\u2003The annotated dataset included 404 positive (9.9%) and 3,686 (90.1%) negative reports. Our model achieved a sensitivity of 92.9% (95% confidence interval: 80.9\u201397.5%), a positive predictive value of 83.0% (69.9\u201391.1)%, a specificity of 97.8% (95.8\u201398.9)%, and an F1 score of 87.6%. We developed a front-end web application based on the model's output.<\/jats:p><jats:p>\n          Conclusion\u2003Developing an NLP-enabled custom web application for tracking and management of high-risk adrenal incidentalomas is feasible in a resource constrained, safety net hospital. Such applications can be used by an institution's quality department or its primary care providers and can easily be generalized to other types of clinical findings.<\/jats:p>","DOI":"10.1055\/s-0040-1715892","type":"journal-article","created":{"date-parts":[[2020,9,16]],"date-time":"2020-09-16T23:00:41Z","timestamp":1600297241000},"page":"606-616","source":"Crossref","is-referenced-by-count":19,"title":["A Web Application for Adrenal Incidentaloma Identification, Tracking, and Management Using Machine Learning"],"prefix":"10.1055","volume":"11","author":[{"given":"Wasif","family":"Bala","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jackson","family":"Steinkamp","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy","family":"Feeney","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Avneesh","family":"Gupta","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abhinav","family":"Sharma","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jake","family":"Kantrowitz","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Kent Hospital, Brown University Alpert Medical School, Warwick, Rhode Island, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Cordella","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Moses","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frederick Thurston","family":"Drake","sequence":"additional","affiliation":[{"name":"Boston Medical Center, One Boston Medical Center Pl, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2020,9,16]]},"reference":[{"issue":"06","key":"ref1","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1056\/NEJMcp065470","article-title":"Clinical practice. The incidentally discovered adrenal mass","volume":"356","author":"W F Young Jr","year":"2007","journal-title":"N Engl J Med"},{"issue":"06","key":"ref2","doi-asserted-by":"crossref","first-page":"851","DOI":"10.1530\/EJE-10-1147","article-title":"AME position statement on adrenal incidentaloma","volume":"164","author":"M Terzolo","year":"2011","journal-title":"Eur J Endocrinol"},{"issue":"01","key":"ref3","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/S0889-8529(05)70122-5","article-title":"Management approaches to adrenal incidentalomas. A view from Rochester, Minnesota","volume":"29","author":"W F Young Jr","year":"2000","journal-title":"Endocrinol Metab Clin North Am"},{"issue":"02","key":"ref4","first-page":"637","article-title":"A survey on adrenal incidentaloma in Italy","volume":"85","author":"F Mantero","year":"2000","journal-title":"J Clin Endocrinol Metab"},{"issue":"04","key":"ref5","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1530\/EJE-09-0234","article-title":"Recommended evaluation of adrenal incidentalomas is costly, has high false-positive rates and confers a risk of fatal cancer that is similar to the risk of the adrenal lesion becoming malignant; time for a rethink?","volume":"161","author":"T J Cawood","year":"2009","journal-title":"Eur J Endocrinol"},{"key":"ref6","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1016\/j.jacr.2017.11.022","article-title":"Determining adherence to follow-up imaging recommendations","volume":"15","author":"T Mabotuwana","year":"2018","journal-title":"J Am Coll Radiol"},{"issue":"03","key":"ref7","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.jacr.2007.09.003","article-title":"Natural language processing using online analytic processing for assessing recommendations in radiology reports","volume":"5","author":"P A Dang","year":"2008","journal-title":"J Am Coll Radiol"},{"issue":"02","key":"ref8","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1148\/radiol.2532090200","article-title":"Recommendations for additional imaging in radiology reports: multifactorial analysis of 5.9 million examinations","volume":"253","author":"C L Sistrom","year":"2009","journal-title":"Radiology"},{"issue":"03","key":"ref9","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1148\/radiol.2343031536","article-title":"Utilization of radiology services in the United States: levels and trends in modalities, regions, and populations","volume":"234","author":"M Bhargavan","year":"2005","journal-title":"Radiology"},{"issue":"02","key":"ref10","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1055\/s-0040-1708530","article-title":"Preventing diagnostic errors in ambulatory care: an electronic notification tool for incomplete radiology tests","volume":"11","author":"S N Weingart","year":"2020","journal-title":"Appl Clin Inform"},{"issue":"02","key":"ref11","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1055\/s-0038-1654700","article-title":"Semiautomated system for nonurgent, clinically significant pathology results","volume":"9","author":"S D O'Connor","year":"2018","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref12","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1007\/BF03344099","article-title":"Prevalence of adrenal incidentaloma in a contemporary computerized tomography series","volume":"29","author":"S Bovio","year":"2006","journal-title":"J Endocrinol Invest"},{"key":"ref13","volume-title":"An Aging Nation: The Older Population in the United States","author":"J M Ortman","year":"2014"},{"issue":"05","key":"ref14","doi-asserted-by":"crossref","first-page":"424","DOI":"10.7326\/0003-4819-138-5-200303040-00013","article-title":"Management of the clinically inapparent adrenal mass (\u201cincidentaloma\u201d)","volume":"138","author":"M M Grumbach","year":"2003","journal-title":"Ann Intern Med"},{"issue":"09","key":"ref15","doi-asserted-by":"crossref","first-page":"4106","DOI":"10.1210\/jc.2010-0457","article-title":"Approach to the patient with an adrenal incidentaloma","volume":"95","author":"L K Nieman","year":"2010","journal-title":"J Clin Endocrinol Metab"},{"issue":"S 01","key":"ref16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4158\/EP.15.S1.1","article-title":"The American Association of Clinical Endocrinologists and American Association of Endocrine Surgeons medical guidelines for the management of adrenal incidentalomas","volume":"15","author":"M A Zeiger","year":"2009","journal-title":"Endocr Pract"},{"issue":"02","key":"ref17","doi-asserted-by":"crossref","first-page":"G1","DOI":"10.1530\/EJE-16-0467","article-title":"Management of adrenal incidentalomas: European Society of Endocrinology Clinical Practice Guideline in collaboration with the European Network for the Study of Adrenal Tumors","volume":"175","author":"M Fassnacht","year":"2016","journal-title":"Eur J Endocrinol"},{"issue":"04","key":"ref18","doi-asserted-by":"crossref","first-page":"241","DOI":"10.5489\/cuaj.11135","article-title":"Guidelines for the management of the incidentally discovered adrenal mass","volume":"5","author":"A Kapoor","year":"2011","journal-title":"Can Urol Assoc J"},{"issue":"02","key":"ref19","doi-asserted-by":"crossref","first-page":"200","DOI":"10.3803\/EnM.2017.32.2.200","article-title":"Clinical guidelines for the management of adrenal incidentaloma","volume":"32","author":"J M Lee","year":"2017","journal-title":"Endocrinol Metab (Seoul)"},{"issue":"05","key":"ref20","doi-asserted-by":"crossref","first-page":"868","DOI":"10.1016\/j.surg.2019.07.034","article-title":"Evaluation of incidental adrenal masses at a tertiary referral and trauma center","volume":"167","author":"T Feeney","year":"2020","journal-title":"Surgery"},{"issue":"03","key":"ref21","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1007\/s11606-017-4240-6","article-title":"The adrenal incidentaloma: an opportunity to improve patient care","volume":"33","author":"J Becker","year":"2018","journal-title":"J Gen Intern Med"},{"key":"ref23","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.ijmedinf.2019.05.021","article-title":"Identifying incidental findings from radiology reports of trauma patients: an evaluation of automated feature representation methods","volume":"129","author":"G Trivedi","year":"2019","journal-title":"Int J Med Inform"},{"key":"ref25","author":"J Pennington","year":"2014"},{"issue":"01","key":"ref30","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1214\/10-AOS843","article-title":"Rates of convergence in active learning","volume":"39","author":"S Hanneke","year":"2011","journal-title":"Ann Stat"},{"issue":"02","key":"ref31","doi-asserted-by":"crossref","first-page":"e000370","DOI":"10.1136\/bmjoq-2018-000370","article-title":"Process improvement for follow-up radiology report recommendations of lung nodules","volume":"8","author":"P S Lim","year":"2019","journal-title":"BMJ Open Qual"},{"issue":"02","key":"ref32","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1148\/radiol.16142770","article-title":"Natural language processing in radiology: a systematic review","volume":"279","author":"E Pons","year":"2016","journal-title":"Radiology"},{"issue":"05","key":"ref33","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1006\/cbmr.1993.1033","article-title":"Monitoring free-text data using medical language processing","volume":"26","author":"D Zingmond","year":"1993","journal-title":"Comput Biomed Res"},{"issue":"05","key":"ref34","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1016\/j.jbi.2013.06.014","article-title":"Semi-supervised clinical text classification with Laplacian SVMs: an application to cancer case management","volume":"46","author":"V Garla","year":"2013","journal-title":"J Biomed Inform"},{"issue":"02","key":"ref35","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.annemergmed.2013.02.001","article-title":"Automated detection using natural language processing of radiologists recommendations for additional imaging of incidental findings","volume":"62","author":"S Dutta","year":"2013","journal-title":"Ann Emerg Med"},{"key":"ref36","first-page":"1593","article-title":"Automatic identification of critical follow-up recommendation sentences in radiology reports","volume":"2011","author":"M Yetisgen-Yildiz","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"key":"ref37","doi-asserted-by":"crossref","first-page":"103169","DOI":"10.1016\/j.jbi.2019.103169","article-title":"Assisting radiologists with reporting urgent findings to referring physicians: a machine learning approach to identify cases for prompt communication","volume":"93","author":"X Meng","year":"2019","journal-title":"J Biomed Inform"},{"key":"ref38","first-page":"1546","article-title":"Understanding urgency in radiology reporting: identifying associations between clinical findings in radiology reports and their prompt communication to referring physicians","volume":"264","author":"X Meng","year":"2019","journal-title":"Stud Health Technol Inform"},{"key":"ref39","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1186\/1471-2105-15-266","article-title":"Natural language processing of radiology reports for the detection of thromboembolic diseases and clinically relevant incidental findings","volume":"15","author":"A-D Pham","year":"2014","journal-title":"BMC Bioinformatics"},{"issue":"03","key":"ref40","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1007\/s10278-016-9931-8","article-title":"Characterization of change and significance for clinical findings in radiology reports through natural language processing","volume":"30","author":"S Hassanpour","year":"2017","journal-title":"J Digit Imaging"},{"key":"ref41","first-page":"1090","article-title":"Extracting follow-up recommendations and associated anatomy from radiology reports","volume":"245","author":"T Mabotuwana","year":"2017","journal-title":"Stud Health Technol Inform"},{"issue":"01","key":"ref42","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/s10278-019-00260-w","article-title":"Determining follow-up imaging study using radiology reports","volume":"33","author":"S Dalal","year":"2020","journal-title":"J Digit Imaging"},{"issue":"04","key":"ref43","doi-asserted-by":"crossref","first-page":"1051","DOI":"10.4338\/ACI-2016-08-RA-0129","article-title":"Identification of long bone fractures in radiology reports using natural language processing to support healthcare quality improvement","volume":"7","author":"R W Grundmeier","year":"2016","journal-title":"Appl Clin Inform"},{"issue":"03","key":"ref44","doi-asserted-by":"crossref","first-page":"600","DOI":"10.4338\/ACI-2014-11-RA-0110","article-title":"Natural language processing techniques for extracting and categorizing finding measurements in narrative radiology reports","volume":"6","author":"M Sevenster","year":"2015","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref45","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1055\/s-0039-1695791","article-title":"Interactive NLP in clinical care: identifying incidental findings in radiology reports","volume":"10","author":"G Trivedi","year":"2019","journal-title":"Appl Clin Inform"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0040-1715892.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,9,16]],"date-time":"2020-09-16T23:01:48Z","timestamp":1600297308000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0040-1715892"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":39,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2020,8,5]]},"published-print":{"date-parts":[[2020,8]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0040-1715892","relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8]]}}}