{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T16:38:12Z","timestamp":1776271092137,"version":"3.50.1"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T00:00:00Z","timestamp":1680825600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T00:00:00Z","timestamp":1680825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Acute intracranial haemorrhage (AIH) is a potentially life-threatening emergency that requires prompt and accurate assessment and management. This study aims to develop and validate an artificial intelligence (AI) algorithm for diagnosing AIH using brain-computed tomography (CT) images. A retrospective, multi-reader, pivotal, crossover, randomised study was performed to validate the performance of an AI algorithm was trained using 104,666 slices from 3010 patients. Brain CT images (12,663 slices from 296 patients) were evaluated by nine reviewers belonging to one of the three subgroups (non-radiologist physicians, <jats:italic>n<\/jats:italic>\u2009=\u20093; board-certified radiologists, <jats:italic>n<\/jats:italic>\u2009=\u20093; and neuroradiologists, <jats:italic>n<\/jats:italic>\u2009=\u20093) with and without the aid of our AI algorithm. Sensitivity, specificity, and accuracy were compared between AI-unassisted and AI-assisted interpretations using the chi-square test. Brain CT interpretation with AI assistance results in significantly higher diagnostic accuracy than that without AI assistance (0.9703 vs. 0.9471, <jats:italic>p<\/jats:italic>\u2009&lt;\u20090.0001, patient-wise). Among the three subgroups of reviewers, non-radiologist physicians demonstrate the greatest improvement in diagnostic accuracy for brain CT interpretation with AI assistance compared to that without AI assistance. For board-certified radiologists, the diagnostic accuracy for brain CT interpretation is significantly higher with AI assistance than without AI assistance. For neuroradiologists, although brain CT interpretation with AI assistance results in a trend for higher diagnostic accuracy compared to that without AI assistance, the difference does not reach statistical significance. For the detection of AIH, brain CT interpretation with AI assistance results in better diagnostic performance than that without AI assistance, with the most significant improvement observed for non-radiologist physicians.<\/jats:p>","DOI":"10.1038\/s41746-023-00798-8","type":"journal-article","created":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T05:02:37Z","timestamp":1680843757000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["Deep learning based automatic detection algorithm for acute intracranial haemorrhage: a pivotal randomized clinical trial"],"prefix":"10.1038","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8441-4574","authenticated-orcid":false,"given":"Tae Jin","family":"Yun","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2396-4705","authenticated-orcid":false,"given":"Jin Wook","family":"Choi","sequence":"additional","affiliation":[]},{"given":"Miran","family":"Han","sequence":"additional","affiliation":[]},{"given":"Woo Sang","family":"Jung","sequence":"additional","affiliation":[]},{"given":"Seung Hong","family":"Choi","sequence":"additional","affiliation":[]},{"given":"Roh-Eul","family":"Yoo","sequence":"additional","affiliation":[]},{"given":"In Pyeong","family":"Hwang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,4,7]]},"reference":[{"key":"798_CR1","doi-asserted-by":"publisher","first-page":"1632","DOI":"10.1016\/S0140-6736(09)60371-8","volume":"373","author":"AI Qureshi","year":"2009","unstructured":"Qureshi, A. I., Mendelow, A. D. & Hanley, D. F. Intracerebral haemorrhage. Lancet 373, 1632\u20131644 (2009).","journal-title":"Lancet"},{"key":"798_CR2","doi-asserted-by":"publisher","first-page":"2001","DOI":"10.1161\/STROKEAHA.107.183689","volume":"38","author":"J Broderick","year":"2007","unstructured":"Broderick, J. et al. Guidelines for the management of spontaneous intracerebral hemorrhage in adults: 2007 update: a guideline from the American Heart Association\/American Stroke Association Stroke Council, High Blood Pressure Research Council, and the Quality of Care and Outcomes in Research Interdisciplinary Working Group. Stroke 38, 2001\u20132023 (2007).","journal-title":"Stroke"},{"key":"798_CR3","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/S1474-4422(09)70340-0","volume":"9","author":"CJ van Asch","year":"2010","unstructured":"van Asch, C. J. et al. Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin: a systematic review and meta-analysis. Lancet Neurol. 9, 167\u2013176 (2010).","journal-title":"Lancet Neurol."},{"key":"798_CR4","doi-asserted-by":"publisher","first-page":"1823","DOI":"10.1001\/jama.292.15.1823","volume":"292","author":"CS Kidwell","year":"2004","unstructured":"Kidwell, C. S. et al. Comparison of MRI and CT for detection of acute intracerebral hemorrhage. JAMA 292, 1823\u20131830 (2004).","journal-title":"JAMA"},{"key":"798_CR5","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1016\/S0140-6736(18)31878-6","volume":"392","author":"C Cordonnier","year":"2018","unstructured":"Cordonnier, C., Demchuk, A., Ziai, W. & Anderson, C. S. Intracerebral haemorrhage: current approaches to acute management. Lancet 392, 1257\u20131268 (2018).","journal-title":"Lancet"},{"key":"798_CR6","doi-asserted-by":"publisher","first-page":"883","DOI":"10.1016\/j.emc.2016.06.010","volume":"34","author":"A Morotti","year":"2016","unstructured":"Morotti, A. & Goldstein, J. N. Diagnosis and management of acute intracerebral hemorrhage. Emerg. Med. Clin. North. Am. 34, 883\u2013899 (2016).","journal-title":"Emerg. Med. Clin. North. Am."},{"key":"798_CR7","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-77441-z","volume":"10","author":"JY Lee","year":"2020","unstructured":"Lee, J. Y., Kim, J. S., Kim, T. Y. & Kim, Y. S. Detection and classification of intracranial haemorrhage on CT images using a novel deep-learning algorithm. Sci. Rep. 10, 20546 (2020).","journal-title":"Sci. Rep."},{"key":"798_CR8","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/j.neurobiolaging.2021.04.015","volume":"105","author":"I Hwang","year":"2021","unstructured":"Hwang, I. et al. Prediction of brain age from routine T2-weighted spin-echo brain magnetic resonance images with a deep convolutional neural network. Neurobiol. Aging 105, 78\u201385 (2021).","journal-title":"Neurobiol. Aging"},{"key":"798_CR9","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","volume":"18","author":"A Hosny","year":"2018","unstructured":"Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H. & Aerts, H. Artificial intelligence in radiology. Nat. Rev. Cancer 18, 500\u2013510 (2018).","journal-title":"Nat. Rev. Cancer"},{"key":"798_CR10","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1038\/s41746-017-0015-z","volume":"1","author":"MR Arbabshirani","year":"2018","unstructured":"Arbabshirani, M. R. et al. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit. Med. 1, 9 (2018).","journal-title":"NPJ Digit. Med."},{"key":"798_CR11","doi-asserted-by":"publisher","first-page":"2388","DOI":"10.1016\/S0140-6736(18)31645-3","volume":"392","author":"S Chilamkurthy","year":"2018","unstructured":"Chilamkurthy, S. et al. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet 392, 2388\u20132396 (2018).","journal-title":"Lancet"},{"key":"798_CR12","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1007\/s00234-019-02330-w","volume":"62","author":"DT Ginat","year":"2020","unstructured":"Ginat, D. T. Analysis of head CT scans flagged by deep learning software for acute intracranial hemorrhage. Neuroradiology 62, 335\u2013340 (2020).","journal-title":"Neuroradiology"},{"key":"798_CR13","doi-asserted-by":"publisher","first-page":"22737","DOI":"10.1073\/pnas.1908021116","volume":"116","author":"W Kuo","year":"2019","unstructured":"Kuo, W., Hne, C., Mukherjee, P., Malik, J. & Yuh, E. L. Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning. Proc. Natl. Acad. Sci. USA 116, 22737\u201322745 (2019).","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"798_CR14","doi-asserted-by":"publisher","first-page":"2","DOI":"10.3174\/ajnr.A6883","volume":"42","author":"JE Soun","year":"2021","unstructured":"Soun, J. E. et al. Artificial intelligence and acute stroke imaging. Am. J. Neuroradiol. 42, 2\u201311 (2021).","journal-title":"Am. J. Neuroradiol."},{"key":"798_CR15","doi-asserted-by":"publisher","first-page":"6191","DOI":"10.1007\/s00330-019-06163-2","volume":"29","author":"H Ye","year":"2019","unstructured":"Ye, H. et al. Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. Eur. Radiol. 29, 6191\u20136201 (2019).","journal-title":"Eur. Radiol."},{"key":"798_CR16","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1016\/j.oret.2017.03.015","volume":"2","author":"U Schmidt-Erfurth","year":"2018","unstructured":"Schmidt-Erfurth, U. et al. Machine learning to analyze the prognostic value of current imaging biomarkers in neovascular age-related macular degeneration. Ophthalmol. Retina 2, 24\u201330 (2018).","journal-title":"Ophthalmol. Retina"},{"key":"798_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms4797","volume":"5","author":"Z Wang","year":"2014","unstructured":"Wang, Z. et al. Non-invasive classification of microcalcifications with phase-contrast X-ray mammography. Nat. Commun. 5, 3797 (2014).","journal-title":"Nat. Commun."},{"key":"798_CR18","unstructured":"Fernando, T., Gammulle, H., Denman, S., Sridharan, S. & Fookes, C. J. a. e.-p. Deep learning for medical anomaly detection - a survey. https:\/\/arxiv.org\/abs\/2012.02364 (2020)."},{"key":"798_CR19","doi-asserted-by":"publisher","first-page":"3416","DOI":"10.1161\/STROKEAHA.119.026561","volume":"50","author":"N Ironside","year":"2019","unstructured":"Ironside, N. et al. Fully automated segmentation algorithm for hematoma volumetric analysis in spontaneous intracerebral hemorrhage. Stroke 50, 3416\u20133423 (2019).","journal-title":"Stroke"},{"key":"798_CR20","doi-asserted-by":"publisher","first-page":"106936","DOI":"10.1016\/j.jmr.2021.106936","volume":"325","author":"J Jang","year":"2021","unstructured":"Jang, J., Lee, H. H., Park, J. A. & Kim, H. Unsupervised anomaly detection using generative adversarial networks in (1)H-MRS of the brain. J. Magn. Reson. 325, 106936 (2021).","journal-title":"J. Magn. Reson."},{"key":"798_CR21","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/j.media.2019.01.010","volume":"54","author":"T Schlegl","year":"2019","unstructured":"Schlegl, T., Seebock, P., Waldstein, S. M., Langs, G. & Schmidt-Erfurth, U. f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54, 30\u201344 (2019).","journal-title":"Med. Image Anal."},{"key":"798_CR22","unstructured":"Schlegl, T., Seeb\u00f6ck, P., Waldstein, S. M., Schmidt-Erfurth, U. & Langs, G. J. a. e.-p. In: Information Processing in Medical Imaging. https:\/\/arxiv.org\/abs\/1703.05921 (2017)."},{"key":"798_CR23","doi-asserted-by":"publisher","first-page":"4629859","DOI":"10.1155\/2019\/4629859","volume":"2019","author":"AM Dawud","year":"2019","unstructured":"Dawud, A. M., Yurtkan, K. & Oztoprak, H. Application of deep learning in neuroradiology: brain haemorrhage classification using transfer learning. Comput. Intell Neurosci. 2019, 4629859 (2019).","journal-title":"Comput. Intell Neurosci."},{"key":"798_CR24","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1038\/s41551-018-0324-9","volume":"3","author":"H Lee","year":"2019","unstructured":"Lee, H. et al. An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets. Nat. Biomed. Eng. 3, 173\u2013182 (2019).","journal-title":"Nat. Biomed. Eng."},{"key":"798_CR25","doi-asserted-by":"publisher","first-page":"923","DOI":"10.1148\/radiol.2017162664","volume":"285","author":"LM Prevedello","year":"2017","unstructured":"Prevedello, L. M. et al. Automated critical test findings identification and online notification system using artificial intelligence in imaging. Radiology 285, 923\u2013931 (2017).","journal-title":"Radiology"},{"key":"798_CR26","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1038\/s41591-018-0147-y","volume":"24","author":"JJ Titano","year":"2018","unstructured":"Titano, J. J. et al. Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat. Med. 24, 1337\u20131341 (2018).","journal-title":"Nat. Med."},{"key":"798_CR27","unstructured":"Grewal, M., Srivastava, M. M., Kumar, P. & Varadarajan, S. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) 281\u2013284 (2018)."},{"key":"798_CR28","doi-asserted-by":"publisher","unstructured":"Mauri, L. & Damiani, E. Estimating degradation of machine learning data assets. ACM J. 14, https:\/\/doi.org\/10.1145\/3446331 (2022).","DOI":"10.1145\/3446331"},{"key":"798_CR29","first-page":"10070","volume":"2","author":"Z Young","year":"2022","unstructured":"Young, Z. & Steele, R. Empirical evaluation of performance degradation of machine learning-based predictive models\u2013a case study in healthcare information systems. Int. J. Inf. Manag. 2, 10070 (2022).","journal-title":"Int. J. Inf. Manag."},{"key":"798_CR30","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"798_CR31","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1016\/j.media.2016.07.007","volume":"35","author":"T Kooi","year":"2017","unstructured":"Kooi, T. et al. Large scale deep learning for computer aided detection of mammographic lesions. Med. Image. Anal. 35, 303\u2013312 (2017).","journal-title":"Med. Image. Anal."},{"key":"798_CR32","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1016\/j.knosys.2017.10.009","volume":"139","author":"LW Zhang","year":"2018","unstructured":"Zhang, L. W., Lin, J. & Karim, R. Adaptive kernel density-based anomaly detection for nonlinear systems. Knowl. Based Syst. 139, 50\u201363 (2018).","journal-title":"Knowl. Based Syst."},{"key":"798_CR33","first-page":"1855","volume":"40","author":"JG Fletcher","year":"2019","unstructured":"Fletcher, J. G. et al. Evaluation of lower-dose spiral head CT for detection of intracranial findings causing neurologic deficits. Am. J. Neuroradiol. 40, 1855\u20131863 (2019).","journal-title":"Am. J. Neuroradiol."},{"key":"798_CR34","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1148\/radiol.2015141991","volume":"276","author":"JG Fletcher","year":"2015","unstructured":"Fletcher, J. G. et al. Observer performance in the detection and classification of malignant hepatic nodules and masses with ct image-space denoising and iterative reconstruction. Radiology 276, 465\u2013478 (2015).","journal-title":"Radiology"},{"key":"798_CR35","doi-asserted-by":"publisher","unstructured":"Sage, A. & Badura, P. Intracranial hemorrhage detection in head CT using double-branch convolutional neural network, support vector machine, and random forest. Appl. Sci. 10, https:\/\/doi.org\/10.3390\/app10217577 (2020).","DOI":"10.3390\/app10217577"},{"key":"798_CR36","unstructured":"Kingma, D. P. & Welling, M. In: International Conference on Learning Representations. https:\/\/arxiv.org\/abs\/1312.6114 (2013)."},{"key":"798_CR37","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1561\/2200000056","volume":"12","author":"DP Kingma","year":"2019","unstructured":"Kingma, D. P. & Welling, M. An introduction to variational autoencoders. Found Trends Mach. Learn. 12, 4\u201389 (2019).","journal-title":"Found Trends Mach. Learn."},{"key":"798_CR38","unstructured":"Goodfellow, I. J. et al. In: Neural Information Processing Systems. https:\/\/arxiv.org\/abs\/1406.2661 (2014)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00798-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00798-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00798-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T05:08:34Z","timestamp":1680844114000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00798-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,7]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["798"],"URL":"https:\/\/doi.org\/10.1038\/s41746-023-00798-8","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,7]]},"assertion":[{"value":"10 September 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 April 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"61"}}