{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T04:04:24Z","timestamp":1790222664172,"version":"4.1.0"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T00:00:00Z","timestamp":1616457600000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Vysioneer Inc"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>Stereotactic radiosurgery (SRS), a validated treatment for brain tumors, requires accurate tumor contouring. This manual segmentation process is time-consuming and prone to substantial inter-practitioner variability. Artificial intelligence (AI) with deep neural networks have increasingly been proposed for use in lesion detection and segmentation but have seldom been validated in a clinical setting.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>We conducted a randomized, cross-modal, multi-reader, multispecialty, multi-case study to evaluate the impact of AI assistance on brain tumor SRS. A state-of-the-art auto-contouring algorithm built on multi-modality imaging and ensemble neural networks was integrated into the clinical workflow. Nine medical professionals contoured the same case series in two reader modes (assisted or unassisted) with a memory washout period of 6 weeks between each section. The case series consisted of 10 algorithm-unseen cases, including five cases of brain metastases, three of meningiomas, and two of acoustic neuromas. Among the nine readers, three experienced experts determined the ground truths of tumor contours.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>With the AI assistance, the inter-reader agreement significantly increased (Dice similarity coefficient [DSC] from 0.86 to 0.90, P &amp;lt; 0.001). Algorithm-assisted physicians demonstrated a higher sensitivity for lesion detection than unassisted physicians (91.3% vs 82.6%, P = .030). AI assistance improved contouring accuracy, with an average increase in DSC of 0.028, especially for physicians with less SRS experience (average DSC from 0.847 to 0.865, P = .002). In addition, AI assistance improved efficiency with a median of 30.8% time-saving. Less-experienced clinicians gained prominent improvement on contouring accuracy but less benefit in reduction of working hours. By contrast, SRS specialists had a relatively minor advantage in DSC, but greater time-saving with the aid of AI.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Deep learning neural networks can be optimally utilized to improve accuracy and ef\ufb01ciency for the clinical workflow in brain tumor SRS.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/neuonc\/noab071","type":"journal-article","created":{"date-parts":[[2021,3,19]],"date-time":"2021-03-19T20:23:59Z","timestamp":1616185439000},"page":"1560-1568","source":"Crossref","is-referenced-by-count":61,"title":["Randomized multi-reader evaluation of automated detection and segmentation of brain tumors in stereotactic radiosurgery with deep neural networks"],"prefix":"10.1093","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7056-1899","authenticated-orcid":false,"given":"Shao-Lun","family":"Lu","sequence":"first","affiliation":[{"name":"Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan"},{"name":"Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fu-Ren","family":"Xiao","sequence":"additional","affiliation":[{"name":"Department of Surgery, National Taiwan University Hospital, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jason Chia-Hsien","family":"Cheng","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan"},{"name":"Graduate Institute of Oncology, National Taiwan University College of Medicine, Taipei, Taiwan"},{"name":"Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wen-Chi","family":"Yang","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan"},{"name":"Graduate Institute of Oncology, National Taiwan University College of Medicine, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yueh-Hung","family":"Cheng","sequence":"additional","affiliation":[{"name":"Vysioneer Inc., Cambridge, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu-Cheng","family":"Chang","sequence":"additional","affiliation":[{"name":"Vysioneer Inc., Cambridge, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jhih-Yuan","family":"Lin","sequence":"additional","affiliation":[{"name":"Vysioneer Inc., Cambridge, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chih-Hung","family":"Liang","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jen-Tang","family":"Lu","sequence":"additional","affiliation":[{"name":"Vysioneer Inc., Cambridge, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ya-Fang","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Medical Imaging, National Taiwan University Hospital, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9943-6039","authenticated-orcid":false,"given":"Feng-Ming","family":"Hsu","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan"},{"name":"Graduate Institute of Oncology, National Taiwan University College of Medicine, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,3,22]]},"reference":[{"issue":"4","key":"2021090111021352000_CIT0001","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1016\/S1474-4422(18)30468-X","article-title":"Global, regional, and national burden of brain and other CNS cancer, 1990\u20132016: a systematic analysis for the Global Burden of Disease Study 2016","volume":"18","author":"Patel","year":"2019","journal-title":"Lancet Neurol."},{"key":"2021090111021352000_CIT0002","author":"American Cancer Society","year":"2020"},{"issue":"1","key":"2021090111021352000_CIT0003","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41572-018-0055-y","article-title":"Brain metastases","volume":"5","author":"Achrol","year":"2019","journal-title":"Nat Rev Dis Primers."},{"key":"2021090111021352000_CIT0004","author":"National Comprehensive Cancer Network"},{"issue":"30","key":"2021090111021352000_CIT0005","doi-asserted-by":"crossref","first-page":"3475","DOI":"10.1200\/JCO.2015.60.9503","article-title":"Treatment of brain metastases","volume":"33","author":"Lin","year":"2015","journal-title":"J Clin Oncol."},{"issue":"1","key":"2021090111021352000_CIT0006","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.clon.2019.06.015","article-title":"Inter-observer variability in target volume delineations of benign and metastatic brain tumours for stereotactic radiosurgery: results of a national quality assurance programme","volume":"32","author":"Growcott","year":"2020","journal-title":"Clin Oncol (R Coll Radiol)."},{"issue":"11","key":"2021090111021352000_CIT0007","doi-asserted-by":"crossref","first-page":"1515","DOI":"10.1080\/0284186X.2018.1473636","article-title":"Multi-institutional study of the variability in target delineation for six targets commonly treated with radiosurgery","volume":"57","author":"Sandstr\u00f6m","year":"2018","journal-title":"Acta Oncol."},{"issue":"7553","key":"2021090111021352000_CIT0008","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature."},{"issue":"1","key":"2021090111021352000_CIT0009","doi-asserted-by":"crossref","first-page":"6268","DOI":"10.1038\/s41598-019-42557-4","article-title":"Efficient deep network architectures for fast chest X-ray tuberculosis screening and visualization","volume":"9","author":"Pasa","year":"2019","journal-title":"Sci Rep."},{"key":"2021090111021352000_CIT0010","doi-asserted-by":"crossref","first-page":"24454","DOI":"10.1038\/srep24454","article-title":"Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans","volume":"6","author":"Cheng","year":"2016","journal-title":"Sci Rep."},{"issue":"7639","key":"2021090111021352000_CIT0011","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature."},{"issue":"22","key":"2021090111021352000_CIT0012","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA."},{"key":"2021090111021352000_CIT0013","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1038\/s41746-019-0112-2","article-title":"Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer","volume":"2","author":"Nagpal","year":"2019","journal-title":"NPJ Digit Med."},{"issue":"12","key":"2021090111021352000_CIT0014","doi-asserted-by":"crossref","first-page":"1636","DOI":"10.1097\/PAS.0000000000001151","article-title":"Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer","volume":"42","author":"Steiner","year":"2018","journal-title":"Am J Surg Pathol."},{"issue":"2","key":"2021090111021352000_CIT0015","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s00345-020-03214-y","article-title":"Early experience with Watson for Oncology: a clinical decision-support system for prostate cancer treatment recommendations","volume":"39","author":"Yu","year":"2021","journal-title":"World J Urol."},{"issue":"7788","key":"2021090111021352000_CIT0016","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1038\/s41586-019-1799-6","article-title":"International evaluation of an AI system for breast cancer screening","volume":"577","author":"McKinney","year":"2020","journal-title":"Nature."},{"key":"2021090111021352000_CIT0017","article-title":"Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy","author":"Nikolov","year":"2018"},{"key":"2021090111021352000_CIT0018","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46723-8_19","volume-title":"Deep Neural Networks for Fast Segmentation of 3D Medical Images","author":"Fritscher","year":"2016"},{"key":"2021090111021352000_CIT0019","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1016\/j.ejmp.2019.12.008","article-title":"Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network","volume":"69","author":"Liu","year":"2020","journal-title":"Phys Med."},{"issue":"2","key":"2021090111021352000_CIT0020","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.radonc.2017.11.012","article-title":"Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer","volume":"126","author":"Lustberg","year":"2018","journal-title":"Radiother Oncol."},{"key":"2021090111021352000_CIT0021","doi-asserted-by":"crossref","first-page":"144591","DOI":"10.1109\/ACCESS.2019.2944958","article-title":"Organ at risk segmentation in head and neck ct images using a two-stage segmentation framework based on 3D U-Net","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access."},{"issue":"3","key":"2021090111021352000_CIT0022","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1148\/radiol.2019182012","article-title":"Deep learning for automated contouring of primary tumor volumes by MRI for nasopharyngeal Carcinoma","volume":"291","author":"Lin","year":"2019","journal-title":"Radiology."},{"issue":"12","key":"2021090111021352000_CIT0023","doi-asserted-by":"crossref","first-page":"6377","DOI":"10.1002\/mp.12602","article-title":"Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks","volume":"44","author":"Men","year":"2017","journal-title":"Med Phys."},{"key":"2021090111021352000_CIT0024","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.radonc.2020.01.021","article-title":"Auto-segmentation of pancreatic tumor in multi-parametric MRI using deep convolutional neural networks","volume":"145","author":"Liang","year":"2020","journal-title":"Radiother Oncol."},{"issue":"20","key":"2021090111021352000_CIT0025","doi-asserted-by":"crossref","first-page":"205015","DOI":"10.1088\/1361-6560\/ab440d","article-title":"Gross tumor volume segmentation for head and neck cancer radiotherapy using deep dense multi-modality network","volume":"64","author":"Guo","year":"2019","journal-title":"Phys Med Biol."},{"key":"2021090111021352000_CIT0026","doi-asserted-by":"crossref","article-title":"Multimodal volume-aware detection and segmentation for brain metastases radiosurgery","author":"Hu","DOI":"10.1007\/978-3-030-32486-5_8"},{"issue":"10","key":"2021090111021352000_CIT0027","doi-asserted-by":"crossref","first-page":"e0185844","DOI":"10.1371\/journal.pone.0185844","article-title":"A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery","volume":"12","author":"Liu","year":"2017","journal-title":"PLoS One."},{"issue":"1","key":"2021090111021352000_CIT0028","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1002\/jmri.26766","article-title":"Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI","volume":"51","author":"Gr\u00f8vik","year":"2020","journal-title":"J Magn Reson Imaging."},{"issue":"4","key":"2021090111021352000_CIT0029","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1093\/neuonc\/noz234","article-title":"Deep learning-based detection and segmentation-assisted management of brain metastases","volume":"22","author":"Xue","year":"2020","journal-title":"Neuro Oncol."},{"issue":"1","key":"2021090111021352000_CIT0030","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1186\/s13014-020-01514-6","article-title":"Deep convolutional neural networks for automated segmentation of brain metastases trained on clinical data","volume":"15","author":"Bousabarah","year":"2020","journal-title":"Radiat Oncol."},{"key":"2021090111021352000_CIT0031","doi-asserted-by":"crossref","first-page":"56","DOI":"10.3389\/fncom.2019.00056","article-title":"Automatic brain tumor segmentation based on cascaded convolutional neural networks with uncertainty estimation","volume":"13","author":"Wang","year":"2019","journal-title":"Front Comput Neurosci."},{"key":"2021090111021352000_CIT0032","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.compbiomed.2018.02.004","article-title":"Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network","volume":"95","author":"Charron","year":"2018","journal-title":"Comput Biol Med."},{"issue":"12","key":"2021090111021352000_CIT0033","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1038\/s41571-020-0417-8","article-title":"Artificial intelligence in radiation oncology","volume":"17","author":"Huynh","year":"2020","journal-title":"Nat Rev Clin Oncol."},{"key":"2021090111021352000_CIT0034","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.media.2016.10.004","article-title":"Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation","volume":"36","author":"Kamnitsas","year":"2017","journal-title":"Med Image Anal."},{"key":"2021090111021352000_CIT0035","doi-asserted-by":"crossref","article-title":"3D U-Net: learning dense volumetric segmentation from sparse annotation","author":"\u00c7i\u00e7ek","DOI":"10.1007\/978-3-319-46723-8_49"},{"key":"2021090111021352000_CIT0036","doi-asserted-by":"crossref","article-title":"Ensemble methods in machine learning","author":"Dietterich","DOI":"10.1007\/3-540-45014-9_1"},{"key":"2021090111021352000_CIT0037","doi-asserted-by":"crossref","article-title":"SimpleElastix: a user-friendly, multi-lingual library for medical image registration","author":"Marstal","DOI":"10.1109\/CVPRW.2016.78"},{"issue":"2","key":"2021090111021352000_CIT0038","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1120\/jacmp.v10i2.2906","article-title":"A comparative study of three CT and MRI registration algorithms in nasopharyngeal carcinoma","volume":"10","author":"Wang","year":"2009","journal-title":"J Appl Clin Med Phys."},{"issue":"5","key":"2021090111021352000_CIT0039","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1016\/j.ijrobp.2013.09.013","article-title":"The effect of contouring variability on dosimetric parameters for brain metastases treated with stereotactic radiosurgery","volume":"87","author":"Stanley","year":"2013","journal-title":"Int J Radiat Oncol Biol Phys."},{"key":"2021090111021352000_CIT0040","author":"Allyn","year":"2020"},{"issue":"12","key":"2021090111021352000_CIT0041","doi-asserted-by":"crossref","first-page":"1494","DOI":"10.6004\/jnccn.2017.7003","article-title":"Radiosurgery for brain metastases: changing practice patterns and disparities in the United States","volume":"15","author":"Kann","year":"2017","journal-title":"J Natl Compr Canc Netw."}],"container-title":["Neuro-Oncology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/neuro-oncology\/advance-article-pdf\/doi\/10.1093\/neuonc\/noab071\/38197058\/noab071.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/neuro-oncology\/article-pdf\/23\/9\/1560\/40186786\/noab071.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/neuro-oncology\/article-pdf\/23\/9\/1560\/40186786\/noab071.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T11:03:16Z","timestamp":1630494196000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article\/23\/9\/1560\/6180141"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,22]]},"references-count":41,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,3,22]]},"published-print":{"date-parts":[[2021,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/neuonc\/noab071","relation":{},"ISSN":["1522-8517","1523-5866"],"issn-type":[{"value":"1522-8517","type":"print"},{"value":"1523-5866","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,9,1]]},"published":{"date-parts":[[2021,3,22]]}}}