{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T11:45:18Z","timestamp":1794051918708,"version":"4.8.0"},"reference-count":14,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurol."],"abstract":"<jats:p><jats:bold>Purpose:<\/jats:bold> Recently developed machine-learning algorithms have demonstrated strong performance in the detection of intracranial hemorrhage (ICH) and large vessel occlusion (LVO). However, their generalizability is often limited by geographic bias of studies. The aim of this study was to validate a commercially available deep learning-based tool in the detection of both ICH and LVO across multiple hospital sites and vendors throughout the U.S.<\/jats:p><jats:p><jats:bold>Materials and Methods:<\/jats:bold> This was a retrospective and multicenter study using anonymized data from two institutions. Eight hundred fourteen non-contrast CT cases and 378 CT angiography cases were analyzed to evaluate ICH and LVO, respectively. The tool's ability to detect and quantify ICH, LVO, and their various subtypes was assessed among multiple CT vendors and hospitals across the United States. Ground truth was based off imaging interpretations from two board-certified neuroradiologists.<\/jats:p><jats:p><jats:bold>Results:<\/jats:bold> There were 255 positive and 559 negative ICH cases. Accuracy was 95.6%, sensitivity was 91.4%, and specificity was 97.5% for the ICH tool. ICH was further stratified into the following subtypes: intraparenchymal, intraventricular, epidural\/subdural, and subarachnoid with true positive rates of 92.9, 100, 94.3, and 89.9%, respectively. ICH true positive rates by volume [small (&amp;lt;5 mL), medium (5\u201325 mL), and large (&amp;gt;25 mL)] were 71.8, 100, and 100%, respectively. There were 156 positive and 222 negative LVO cases. The LVO tool demonstrated an accuracy of 98.1%, sensitivity of 98.1%, and specificity of 98.2%. A subset of 55 randomly selected cases were also assessed for LVO detection at various sites, including the distal internal carotid artery, middle cerebral artery M1 segment, proximal middle cerebral artery M2 segment, and distal middle cerebral artery M2 segment with an accuracy of 97.0%, sensitivity of 94.3%, and specificity of 97.4%.<\/jats:p><jats:p><jats:bold>Conclusion:<\/jats:bold> Deep learning tools can be effective in the detection of both ICH and LVO across a wide variety of hospital systems. While some limitations were identified, specifically in the detection of small ICH and distal M2 occlusion, this study highlights a deep learning tool that can assist radiologists in the detection of emergent findings in a variety of practice settings.<\/jats:p>","DOI":"10.3389\/fneur.2021.656112","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T09:44:58Z","timestamp":1619689498000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":72,"title":["Validation of a Deep Learning Tool in the Detection of Intracranial Hemorrhage and Large Vessel Occlusion"],"prefix":"10.3389","volume":"12","author":[{"given":"Joel","family":"McLouth","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Elstrott","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yasmina","family":"Chaibi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Quenet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter D.","family":"Chang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel S.","family":"Chow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer E.","family":"Soun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2021,4,29]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1534","DOI":"10.1136\/jnnp.2004.055145","article-title":"Long term survival after primary intracerebral haemorrhage: a retrospective population based study","volume":"76","author":"Fogelholm","year":"2005","journal-title":"J Neurol Neurosurg Psychiatry."},{"key":"B2","doi-asserted-by":"publisher","first-page":"651","DOI":"10.3389\/fneur.2017.00651","article-title":"Ischemic strokes due to large-vessel occlusions contribute disproportionately to stroke-related dependence and death: a review","volume":"8","author":"Malhotra","year":"2017","journal-title":"Front Neurol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1161\/HYPERTENSIONAHA.110.154328","article-title":"Lower treatment blood pressure is associated with greatest reduction in hematoma growth after acute intracerebral hemorrhage","volume":"56","author":"Arima","year":"2010","journal-title":"Hypertension."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1279","DOI":"10.1001\/jama.2016.13647","article-title":"Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis","volume":"316","author":"Saver","year":"2016","journal-title":"JAMA."},{"key":"B5","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1007\/s00234-019-02330-w","article-title":"Analysis of head CT scans flagged by deep learning software for acute intracranial hemorrhage","volume":"62","author":"Ginat","year":"2020","journal-title":"Neuroradiology."},{"key":"B6","doi-asserted-by":"publisher","first-page":"2388","DOI":"10.1016\/S0140-6736(18)31645-3","article-title":"Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study","volume":"392","author":"Chilamkurthy","year":"2018","journal-title":"Lancet."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1609","DOI":"10.3174\/ajnr.A5742","article-title":"Hybrid 3D\/2D convolutional neural network for hemorrhage evaluation on head CT","volume":"39","author":"Chang","year":"2018","journal-title":"AJNR Am J Neuroradiol."},{"key":"B8","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1038\/s41551-018-0324-9","article-title":"An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets","volume":"3","author":"Lee","year":"2019","journal-title":"Nat Biomed Eng."},{"key":"B9","doi-asserted-by":"publisher","first-page":"22737","DOI":"10.1073\/pnas.1908021116","article-title":"Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning","volume":"116","author":"Kuo","year":"2019","journal-title":"Proc Natl Acad Sci USA."},{"key":"B10","doi-asserted-by":"publisher","first-page":"AWMP16","DOI":"10.1161\/str.50.suppl_1.WMP16","article-title":"Abstract WMP16: artificial intelligence detection of cerebrovascular large vessel occlusion - nine month, 650 patient evaluation of the diagnostic accuracy and performance of the viz.ai LVO algorithm","volume":"50","author":"Chatterjee","year":"2018","journal-title":"Stroke."},{"key":"B11","doi-asserted-by":"publisher","first-page":"A101","DOI":"10.1136\/neurintsurg-2018-SNIS.184","article-title":"E-108 Aladin study: automated large artery occlusion detection in stroke imaging study \u2013 a multicenter analysis","volume":"10","author":"Barreira","year":"2018","journal-title":"J Neurointervent Surg."},{"key":"B12","doi-asserted-by":"publisher","first-page":"247","DOI":"10.3174\/ajnr.A6923","article-title":"Evaluation of artificial intelligence\u2013powered identification of large-vessel occlusions in a comprehensive stroke center","volume":"42","author":"Yahav-Dovrat","year":"2020","journal-title":"Am J Neuroradiol."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1212","DOI":"10.1001\/jama.2020.12067","article-title":"Geographic distribution of US cohorts used to train deep learning algorithms","volume":"324","author":"Kaushal","year":"2020","journal-title":"JAMA."},{"key":"B14","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1038\/s41746-017-0015-z","article-title":"Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration","volume":"1","author":"Arbabshirani","year":"2018","journal-title":"NPJ Digit Med."}],"container-title":["Frontiers in Neurology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fneur.2021.656112\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T09:44:59Z","timestamp":1619689499000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fneur.2021.656112\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,29]]},"references-count":14,"alternative-id":["10.3389\/fneur.2021.656112"],"URL":"https:\/\/doi.org\/10.3389\/fneur.2021.656112","relation":{},"ISSN":["1664-2295"],"issn-type":[{"value":"1664-2295","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,29]]},"article-number":"656112"}}