{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T13:32:17Z","timestamp":1791379937574,"version":"4.3.3"},"reference-count":27,"publisher":"IOP Publishing","issue":"22","license":[{"start":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T00:00:00Z","timestamp":1731542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/page\/copyright"},{"start":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T00:00:00Z","timestamp":1731542400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100007210","name":"varian","doi-asserted-by":"crossref","award":["master research agreement grant"],"award-info":[{"award-number":["master research agreement grant"]}],"id":[{"id":"10.13039\/100007210","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007325","name":"American Association of Physicists in Medicine","doi-asserted-by":"crossref","award":["AAPM\/RSNA Doctoral Graduate Fellowship"],"award-info":[{"award-number":["AAPM\/RSNA Doctoral Graduate Fellowship"]}],"id":[{"id":"10.13039\/100007325","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Phys. Med. Biol."],"published-print":{"date-parts":[[2024,11,21]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>\n                  <jats:italic>Objective.<\/jats:italic> Artificial intelligence (AI) based treatment planning tools are being implemented in clinic. However, human interactions with such AI tools are rarely analyzed. This study aims to comprehend human planner\u2019s interaction with the AI planning tool and incorporate the analysis to improve the existing AI tool. <jats:italic>Approach.<\/jats:italic> An in-house AI tool for whole breast radiation therapy planning was deployed in our institution since 2019, among which 522 patients were included in this study. The AI tool automatically generates fluence maps of the tangential beams to create an <jats:italic>AI plan<\/jats:italic>. Human planner makes fluence edits deemed necessary and after attending physician approval for treatment, it is recorded as <jats:italic>final plan<\/jats:italic>. Manual modification value maps were collected, which is the difference between the <jats:italic>AI-plan<\/jats:italic> and the <jats:italic>final plan<\/jats:italic>. Subsequently, a human-AI interaction (HAI) model using full scale connected U-Net was trained to learn such interactions and perform plan enhancements. The trained HAI model automatically modifies the <jats:italic>AI plan<\/jats:italic> to generate AI-modified plans (<jats:italic>AI-m plan<\/jats:italic>), simulating human editing. Its performance was evaluated against original <jats:italic>AI-plan<\/jats:italic> and <jats:italic>final plan. Main results. AI-m plan<\/jats:italic> showed statistically significant improvement in hotspot control over the <jats:italic>AI plan<\/jats:italic>, with an average of 25.2cc volume reduction in breast V105% (<jats:italic>p<\/jats:italic> = 0.011) and 0.805% decrease in Dmax (<jats:italic>p<\/jats:italic> &lt; .001). It also maintained the same planning target volume (PTV) coverage as the <jats:italic>final plan<\/jats:italic>, demonstrating the model has captured the clinic focus of improving PTV hot spots without degrading coverage. <jats:italic>Significance.<\/jats:italic> The proposed HAI model has demonstrated capability of further enhancing the <jats:italic>AI plan<\/jats:italic> via modeling human-AI tool interactions. This study shows analysis of human interaction with the AI planning tool is a significant step to improve the AI tool.<\/jats:p>","DOI":"10.1088\/1361-6560\/ad8e29","type":"journal-article","created":{"date-parts":[[2024,11,2]],"date-time":"2024-11-02T22:53:41Z","timestamp":1730588021000},"page":"225018","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Understanding and modeling human-AI interaction of artificial intelligence tool in radiation oncology clinic using deep neural network: a feasibility study using three year prospective data"],"prefix":"10.1088","volume":"69","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4559-5701","authenticated-orcid":true,"given":"Dongrong","family":"Yang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-6265-0162","authenticated-orcid":true,"given":"Cameron","family":"Murr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9995-8101","authenticated-orcid":true,"given":"Xinyi","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sua","family":"Yoo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachel","family":"Blitzblau","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susan","family":"McDuff","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Stephens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Q Jackie","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0748-7280","authenticated-orcid":true,"given":"Qiuwen","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Sheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2024,11,14]]},"reference":[{"key":"pmbad8e29bib1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102101","article-title":"A deep learning-based framework for segmenting invisible clinical target volumes with estimated uncertainties for post-operative prostate cancer radiotherapy","volume":"72","author":"Balagopal","year":"2021","journal-title":"Med. Image Anal."},{"key":"pmbad8e29bib2","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1016\/j.meddos.2016.06.005","article-title":"Radiation therapy for breast cancer: literature review","volume":"41","author":"Balaji","year":"2016","journal-title":"Med. Dosim."},{"key":"pmbad8e29bib3","doi-asserted-by":"publisher","first-page":"394","DOI":"10.3322\/caac.21492","article-title":"Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"68","author":"Bray","year":"2018","journal-title":"CA Cancer J. Clin."},{"key":"pmbad8e29bib4","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1016\/j.ijrobp.2018.01.114","article-title":"Deep learning algorithm for auto-delineation of high-risk oropharyngeal clinical target volumes with built-in dice similarity coefficient parameter optimization function","volume":"101","author":"Cardenas","year":"2018","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbad8e29bib5","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1016\/j.radonc.2021.04.019","article-title":"A deep learning-based auto-segmentation system for organs-at-risk on whole-body computed tomography images for radiation therapy","volume":"160","author":"Chen","year":"2021","journal-title":"Radiother. Oncol."},{"key":"pmbad8e29bib6","doi-asserted-by":"publisher","first-page":"1707","DOI":"10.1016\/S0140-6736(11)61629-2","article-title":"Effect of radiotherapy after breast-conserving surgery on 10-year recurrence and 15-year breast cancer death: meta-analysis of individual patient data for 10,801 women in 17 randomised trials","volume":"378","author":"Early Breast Cancer Trialists\u2019 Collaborative G","year":"2011","journal-title":"Lancet"},{"key":"pmbad8e29bib7","doi-asserted-by":"publisher","DOI":"10.1117\/1.JMI.7.1.016502","article-title":"Integrating AI into radiology workflow: levels of research, production, and feedback maturity","volume":"7","author":"Dikici","year":"2020","journal-title":"J. Med. Imaging"},{"key":"pmbad8e29bib8","doi-asserted-by":"publisher","first-page":"1914","DOI":"10.1002\/mp.13433","article-title":"Machine learning for automated quality assurance in radiotherapy: a proof of principle using EPID data description","volume":"46","author":"El Naqa","year":"2019","journal-title":"Med. Phys."},{"key":"pmbad8e29bib9","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1002\/mp.13271","article-title":"Automatic treatment planning based on three-dimensional dose distribution predicted from deep learning technique","volume":"46","author":"Fan","year":"2019","journal-title":"Med. Phys."},{"key":"pmbad8e29bib10","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1016\/j.ijrobp.2013.03.015","article-title":"A knowledge-based approach to improving and homogenizing intensity modulated radiation therapy planning quality among treatment centers: an example application to prostate cancer planning","volume":"87","author":"Good","year":"2013","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbad8e29bib11","doi-asserted-by":"crossref","DOI":"10.1109\/ICASSP40776.2020.9053405","article-title":"UNet 3+: a full-scale connected unet for medical image segmentation","author":"Huang","year":"2020"},{"key":"pmbad8e29bib12","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"pmbad8e29bib13","first-page":"p 392","article-title":"A human-AI collaborative approach for clinical decision making on rehabilitation assessment","author":"Lee","year":"2021"},{"key":"pmbad8e29bib14","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/aba5eb","article-title":"Automatic IMRT planning via static field fluence prediction (AIP-SFFP): a deep learning algorithm for real-time prostate treatment planning","volume":"65","author":"Li","year":"2020","journal-title":"Phys. Med. Biol."},{"key":"pmbad8e29bib15","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1148\/radiol.2019182012","article-title":"Deep learning for automated contouring of primary tumor volumes by mri for nasopharyngeal carcinoma","volume":"291","author":"Lin","year":"2019","journal-title":"Radiology"},{"key":"pmbad8e29bib16","doi-asserted-by":"publisher","first-page":"315","DOI":"10.3389\/fonc.2017.00315","article-title":"Deep deconvolutional neural network for target segmentation of nasopharyngeal cancer in planning computed tomography images","volume":"7","author":"Men","year":"2017","journal-title":"Front. Oncol."},{"key":"pmbad8e29bib17","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.prro.2011.11.012","article-title":"Variation in external beam treatment plan quality: an inter-institutional study of planners and planning systems","volume":"2","author":"Nelms","year":"2012","journal-title":"Pract. Radiat. Oncol."},{"key":"pmbad8e29bib18","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.1016\/j.ijrobp.2020.10.038","article-title":"Automatic segmentation using deep learning to enable online dose optimization during adaptive radiation therapy of cervical cancer","volume":"109","author":"Rigaud","year":"2021","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbad8e29bib19","first-page":"234","article-title":"U-net: convolutional networks for biomedical image segmentation","author":"Ronneberger","year":"2015"},{"key":"pmbad8e29bib20","doi-asserted-by":"publisher","first-page":"750","DOI":"10.3389\/fonc.2019.00750","article-title":"Automatic planning of whole breast radiation therapy using machine learning models","volume":"9","author":"Sheng","year":"2019","journal-title":"Front. Oncol."},{"key":"pmbad8e29bib21","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.radonc.2020.01.028","article-title":"Automated delineation of the clinical target volume using anatomically constrained 3D expansion of the gross tumor volume","volume":"146","author":"Shusharina","year":"2020","journal-title":"Radiother. Oncol."},{"key":"pmbad8e29bib22","doi-asserted-by":"publisher","first-page":"4323","DOI":"10.1118\/1.4953835","article-title":"A mathematical framework for virtual IMRT QA using machine learning","volume":"43","author":"Valdes","year":"2016","journal-title":"Med. Phys."},{"key":"pmbad8e29bib23","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nat. Methods"},{"key":"pmbad8e29bib24","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2021.100672","article-title":"Deep learning-based fluence map prediction for pancreas stereotactic body radiation therapy with simultaneous integrated boost","volume":"6","author":"Wang","year":"2021","journal-title":"Adv. Radiat. Oncol."},{"key":"pmbad8e29bib25","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1016\/j.radonc.2019.10.019","article-title":"Comparing deep learning-based auto-segmentation of organs at risk and clinical target volumes to expert inter-observer variability in radiotherapy planning","volume":"144","author":"Wong","year":"2020","journal-title":"Radiother. Oncol."},{"key":"pmbad8e29bib26","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2020.616721","article-title":"An artificial intelligence-based full-process solution for radiotherapy: a proof of concept study on rectal cancer","volume":"10","author":"Xia","year":"2020","journal-title":"Front. Oncol."},{"key":"pmbad8e29bib27","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2021.100656","article-title":"Clinical experience with machine learning-based automated treatment planning for whole breast radiation therapy","volume":"6","author":"Yoo","year":"2021","journal-title":"Adv. Radiat. Oncol."}],"container-title":["Physics in Medicine &amp; Biology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T07:11:38Z","timestamp":1731568298000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ad8e29"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,14]]},"references-count":27,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2024,11,14]]},"published-print":{"date-parts":[[2024,11,21]]}},"URL":"https:\/\/doi.org\/10.1088\/1361-6560\/ad8e29","relation":{},"ISSN":["0031-9155","1361-6560"],"issn-type":[{"value":"0031-9155","type":"print"},{"value":"1361-6560","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,14]]},"assertion":[{"value":"Understanding and modeling human-AI interaction of artificial intelligence tool in radiation oncology clinic using deep neural network: a feasibility study using three year prospective data","name":"article_title","label":"Article Title"},{"value":"Physics in Medicine & Biology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 Institute of Physics and Engineering in Medicine. All rights, including for text and data mining, AI training, and similar technologies, are reserved.","name":"copyright_information","label":"Copyright Information"},{"value":"2024-06-10","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-11-01","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-11-14","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}