{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T12:10:05Z","timestamp":1787314205922,"version":"build-2736575974"},"reference-count":39,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"9","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>The increasing use of patient portal messages has enhanced patient-provider communication. However, the high volume of these messages has also contributed to physician burnout.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS<\/jats:title>\n                    <jats:p>\n                      Patient-generated portal messages sent to a single cancer center from 2011 to 2023 were extracted. BERTopic, a natural language processing topic modeling technique based on large language models, was optimized. For further categorization, the topic words were labeled using GPT-4, followed by review by two oncologists. Uniform Manifold Approximation and Projection was used for dimensionality reduction and visualizing topics. Message volume changes over time were assessed using a Student's\n                      <jats:italic toggle=\"yes\">t<\/jats:italic>\n                      test.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>\n                      A total of 2,280,851 messages were analyzed. The monthly average number of messages increased from 2,071 in 2012 to 43,430 in 2022 (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      &lt; .001). There was a significant rise in message volume after the COVID-19 pandemic, with a posterior probability of a causal effect of 96.4% (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      = .04). Scheduling-related messages were the most frequent across departments, whereas symptoms and health concerns were second or third most common topics. In medical oncology and surgical oncology, topics on prescriptions and medications were more common compared with radiation oncology and gynecologic oncology. Despite concurrent institutional changes in self-scheduling systems, scheduling-related messages did not decrease over time.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>The substantial increase in patient portal messages, particularly scheduling-related inquiries, underscores the need for streamlined communication to reduce the burden on health care providers. These findings highlight the need for strategies to manage message volume and mitigate physician burnout, laying groundwork for artificial intelligence\u2013driven future triage systems to improve message management and patient care.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-25-00102","type":"journal-article","created":{"date-parts":[[2025,10,1]],"date-time":"2025-10-01T20:00:12Z","timestamp":1759348812000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":3,"title":["Unsupervised Large Language Models to Identify Topics in Cancer Center Patient Portal Messages"],"prefix":"10.1200","author":[{"given":"Ji Hyun","family":"Chang","sequence":"first","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"},{"name":"Department of Radiation Oncology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amir","family":"Ashraf-Ganjouei","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0216-4031","authenticated-orcid":true,"given":"Isabel","family":"Friesner","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryzen","family":"Benson","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1620-6455","authenticated-orcid":true,"given":"Travis","family":"Zack","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9705-0340","authenticated-orcid":true,"given":"Sumi","family":"Sinha","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"},{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3152-1023","authenticated-orcid":true,"given":"Jason","family":"Chan","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"},{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9841-9357","authenticated-orcid":true,"given":"Steve","family":"Braunstein","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"},{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amy","family":"Lin","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lisa","family":"Singer","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"},{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5172-6889","authenticated-orcid":true,"given":"Julian C.","family":"Hong","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA"},{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA"},{"name":"UCSF-UC Berkeley Joint Program in Computational Precision Health, Berkeley, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_7_2_2","doi-asserted-by":"publisher","DOI":"10.4338\/ACI-2014-12-RA-0117"},{"key":"e_1_3_7_3_2","doi-asserted-by":"publisher","DOI":"10.2196\/12779"},{"key":"e_1_3_7_4_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2019.18287"},{"key":"e_1_3_7_5_2","doi-asserted-by":"publisher","DOI":"10.1200\/OP.22.00704"},{"key":"e_1_3_7_6_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocy118"},{"key":"e_1_3_7_7_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2022.24710"},{"key":"e_1_3_7_8_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaa092"},{"key":"e_1_3_7_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2021.12.027"},{"key":"e_1_3_7_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.mayocp.2018.05.014"},{"key":"e_1_3_7_11_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.28780"},{"key":"e_1_3_7_12_2","doi-asserted-by":"publisher","DOI":"10.1097\/01.HMR.0000304493.87898.72"},{"key":"e_1_3_7_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCONS.2017.8250563"},{"key":"e_1_3_7_14_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0044-1800726"},{"key":"e_1_3_7_15_2","unstructured":"all-MiniLM-L6-v2. https:\/\/huggingface.co\/sentence-transformers\/all-MiniLM-L6-v2"},{"key":"e_1_3_7_16_2","doi-asserted-by":"publisher","unstructured":"Devlin J Chang M-W Lee K et al: BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv 10.48550\/arXiv.1810.04805","DOI":"10.48550\/arXiv.1810.04805"},{"key":"e_1_3_7_17_2","doi-asserted-by":"publisher","unstructured":"Grootendorst M: BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv 10.48550\/arXiv.2203.05794","DOI":"10.48550\/arXiv.2203.05794"},{"key":"e_1_3_7_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.eacl-demos.31"},{"key":"e_1_3_7_19_2","unstructured":"tfcausalimpact. https:\/\/github.com\/WillianFuks\/tfcausalimpact"},{"key":"e_1_3_7_20_2","doi-asserted-by":"publisher","DOI":"10.1002\/pon.4382"},{"key":"e_1_3_7_21_2","doi-asserted-by":"publisher","DOI":"10.1200\/EDBK_156120"},{"key":"e_1_3_7_22_2","doi-asserted-by":"publisher","DOI":"10.3389\/fsoc.2022.886498"},{"key":"e_1_3_7_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2023.e13118"},{"key":"e_1_3_7_24_2","first-page":"403","article-title":"What do autistic people discuss on Twitter? An approach using BERTopic modelling","volume":"302","author":"Gabarron E","year":"2023","unstructured":"Gabarron E, Dorronzoro E, Reichenpfader D, et al: What do autistic people discuss on Twitter? An approach using BERTopic modelling. Stud Health Technol Inform 302:403-407, 2023","journal-title":"Stud Health Technol Inform"},{"key":"e_1_3_7_25_2","doi-asserted-by":"publisher","DOI":"10.2196\/41319"},{"key":"e_1_3_7_26_2","doi-asserted-by":"publisher","DOI":"10.1111\/coa.14037"},{"key":"e_1_3_7_27_2","doi-asserted-by":"publisher","DOI":"10.2196\/45249"},{"key":"e_1_3_7_28_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz220"},{"key":"e_1_3_7_29_2","first-page":"00176","article-title":"Now that we don\u2019t talk: Should cancer centers bill for patient portal messages in oncology?","volume":"24","author":"Peppercorn J","year":"2024","unstructured":"Peppercorn J: Now that we don\u2019t talk: Should cancer centers bill for patient portal messages in oncology? JCO Oncol Pract 24:00176, 2024","journal-title":"JCO Oncol Pract"},{"key":"e_1_3_7_30_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocae052"},{"key":"e_1_3_7_31_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.3201"},{"key":"e_1_3_7_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(24)00060-8"},{"key":"e_1_3_7_33_2","doi-asserted-by":"publisher","DOI":"10.1200\/CCI.22.00071"},{"key":"e_1_3_7_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2017.06.004"},{"key":"e_1_3_7_35_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0038-1675814"},{"key":"e_1_3_7_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2017.08.014"},{"key":"e_1_3_7_37_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0277489"},{"key":"e_1_3_7_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.hlpt.2023.100761"},{"key":"e_1_3_7_39_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoncol.2021.0540"},{"key":"e_1_3_7_40_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.20.01688"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-25-00102","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,1]],"date-time":"2025-10-01T20:00:13Z","timestamp":1759348813000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-25-00102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10]]},"references-count":39,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,10]]}},"alternative-id":["10.1200\/CCI-25-00102"],"URL":"https:\/\/doi.org\/10.1200\/cci-25-00102","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10]]},"assertion":[{"value":"2025-04-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-11","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2500102"}}