{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T17:46:25Z","timestamp":1755798385953,"version":"3.44.0"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2025,7,18]],"date-time":"2025-07-18T00:00:00Z","timestamp":1752796800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"Mass General Brigham Institutional Funds"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Real-world data (RWD) analyses primarily rely on structured clinical documentation collected through routine clinical care or driven by medical billing requirements. Patient-reported outcome measures (PROMs), integrated into electronic health records (EHRs), are an additional data source that could offer valuable insights into a patient\u2019s perspective and contribute to a more comprehensive understanding of health outcomes in RWD studies. This study aims to characterize agreement between PROMs symptoms and structured clinical documentation of these symptoms by clinicians in EHRs.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>A cross-sectional study of 913\u2009244 adult primary care annual physical visits between January 1, 2019 and December 31, 2023. We compared differences in prevalence and agreement of patient-reported symptoms (PRS) and structured clinician documentation (CD) across 15 respiratory, gastrointestinal, cardiometabolic, and neuropsychiatric symptoms.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Patient-reported symptom prevalence were significantly higher compared to CD across most symptoms including joint pain (33% PRS vs 12%), headaches (17% PRS vs 8.8% CD), and sleep disturbance (24% PRS vs 10% CD). Clinicians documented anxiety (11% PRS vs 23% CD) and depression (6.6% PRS vs 15.4% CD) symptoms using structured code at higher rates than patients reported them. Agreement between symptom self-report and clinician-documented structured codes was low to moderate (\u03ba: 0.06-0.39).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Primary care patients self-report symptoms up to ten times more frequently than clinicians document them with structured codes in the EHR.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>This work demonstrates the value and feasibility of incorporating PRSs in RWD studies to reduce misclassification and more holistically capture a patient\u2019s health.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaf112","type":"journal-article","created":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T07:31:17Z","timestamp":1751787077000},"page":"1454-1461","source":"Crossref","is-referenced-by-count":0,"title":["Comparing patient-reported symptoms and structured clinician documentation in electronic health records"],"prefix":"10.1093","volume":"32","author":[{"given":"Victor M","family":"Castro","sequence":"first","affiliation":[{"name":"Department of Public Health and Health Sciences, Bouv\u00e9 College of Health Sciences, Northeastern University , Boston, MA, 02115,","place":["United States"]},{"name":"Research Information Science and Computing, Mass General Brigham , Somerville, MA, 02145,","place":["United States"]}]},{"given":"Vivian S","family":"Gainer","sequence":"additional","affiliation":[{"name":"Research Information Science and Computing, Mass General Brigham , Somerville, MA, 02145,","place":["United States"]}]},{"given":"Danielle M","family":"Crookes","sequence":"additional","affiliation":[{"name":"Department of Public Health and Health Sciences, Bouv\u00e9 College of Health Sciences, Northeastern University , Boston, MA, 02115,","place":["United States"]}]},{"given":"Shawn N","family":"Murphy","sequence":"additional","affiliation":[{"name":"Research Information Science and Computing, Mass General Brigham , Somerville, MA, 02145,","place":["United States"]},{"name":"Harvard Medical School , Boston, MA, 02115,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2454-4489","authenticated-orcid":false,"given":"Justin","family":"Manjourides","sequence":"additional","affiliation":[{"name":"Department of Public Health and Health Sciences, Bouv\u00e9 College of Health Sciences, Northeastern University , Boston, MA, 02115,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2025,7,18]]},"reference":[{"key":"2025081903513118800_ocaf112-B1","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1016\/j.jval.2017.03.008","article-title":"What is real-world data? a review of definitions based on literature and stakeholder interviews","volume":"20","author":"Makady","year":"2017","journal-title":"Value Health"},{"key":"2025081903513118800_ocaf112-B2","doi-asserted-by":"crossref","first-page":"e532","DOI":"10.1016\/S2589-7500(22)00048-6","article-title":"Identifying who has long COVID in the USA: a machine learning approach using N3C data","volume":"4","author":"Pfaff","year":"2022","journal-title":"Lancet Digit Health"},{"key":"2025081903513118800_ocaf112-B3","doi-asserted-by":"publisher","first-page":"554","DOI":"10.1001\/jamainternmed.2023.0743","article-title":"Association of treatment with nirmatrelvir and the risk of post\u2013COVID-19 condition","volume":"183","author":"Xie","year":"2023","journal-title":"JAMA Intern Med"},{"key":"2025081903513118800_ocaf112-B4","doi-asserted-by":"crossref","first-page":"2293","DOI":"10.1056\/NEJMsb1609216","article-title":"Real-world evidence\u2014what is it and what can it tell us?","volume":"375","author":"Sherman","year":"2016","journal-title":"N Engl J Med"},{"key":"2025081903513118800_ocaf112-B5","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1001\/jama.2018.10136","article-title":"Real-world evidence and real-world data for evaluating drug safety and effectiveness","volume":"320","author":"Corrigan-Curay","year":"2018","journal-title":"JAMA"},{"key":"2025081903513118800_ocaf112-B6","doi-asserted-by":"crossref","first-page":"e46364","DOI":"10.2196\/46364","article-title":"Determining distinct suicide attempts from recurrent electronic health record codes: classification study","volume":"8","author":"Bentley","year":"2024","journal-title":"JMIR Form Res"},{"key":"2025081903513118800_ocaf112-B7","doi-asserted-by":"crossref","first-page":"e000547","DOI":"10.1136\/bmjdrc-2018-000547","article-title":"Validity of ICD-10-CM codes for determination of diabetes type for persons with youth-onset type 1 and type 2 diabetes","volume":"7","author":"Chi","year":"2019","journal-title":"BMJ Open Diabetes Res Care"},{"author":"Pfaff","key":"2025081903513118800_ocaf112-B8"},{"key":"2025081903513118800_ocaf112-B9","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1136\/bmjqs-2020-012206","article-title":"Primary care physician\u2019s (PCP) perceived value of patient-reported outcomes (PROs) in clinical practice: a mixed methods study","volume":"31","author":"Mou","year":"2022","journal-title":"BMJ Qual Saf"},{"key":"2025081903513118800_ocaf112-B10","doi-asserted-by":"crossref","first-page":"2265","DOI":"10.1093\/jamia\/ocab138","article-title":"Digital disparities: lessons learned from a patient reported outcomes program during the COVID-19 pandemic","volume":"28","author":"Sisodia","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2025081903513118800_ocaf112-B11","doi-asserted-by":"crossref","first-page":"1575","DOI":"10.1007\/s11136-019-02112-0","article-title":"Advancing the use of patient-reported outcomes in practice: understanding challenges, opportunities, and the potential of health information technology","volume":"28","author":"Hsiao","year":"2019","journal-title":"Qual Life Res"},{"key":"2025081903513118800_ocaf112-B12","doi-asserted-by":"crossref","first-page":"f167","DOI":"10.1136\/bmj.f167","article-title":"Patient reported outcome measures could help transform healthcare","volume":"346","author":"Black","year":"2013","journal-title":"BMJ"},{"key":"2025081903513118800_ocaf112-B13","first-page":"530","article-title":"Agreement between patient-reported symptoms and their documentation in the medical record","volume":"14","author":"Pakhomov","year":"2008","journal-title":"Am J Manag Care"},{"key":"2025081903513118800_ocaf112-B14","doi-asserted-by":"crossref","first-page":"145","DOI":"10.4104\/pcrj.2011.00098","article-title":"Agreement between self-report and medical records on signs and symptoms of respiratory illness","volume":"21","author":"Barbara","year":"2012","journal-title":"Prim Care Respir J"},{"key":"2025081903513118800_ocaf112-B15","doi-asserted-by":"crossref","first-page":"1624","DOI":"10.1093\/jnci\/djp386","article-title":"Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes","volume":"101","author":"Basch","year":"2009","journal-title":"J Natl Cancer Inst"},{"key":"2025081903513118800_ocaf112-B16","doi-asserted-by":"crossref","first-page":"1565","DOI":"10.1016\/S0277-9536(00)00269-0","article-title":"Gender differences in descriptions of angina symptoms and health problems immediately prior to angiography: the ACRE study","volume":"52","author":"Philpott","year":"2001","journal-title":"Soc Sci Med"},{"key":"2025081903513118800_ocaf112-B17","doi-asserted-by":"publisher","first-page":"1131","DOI":"10.1093\/jamia\/ocac046","article-title":"Comparing medical history data derived from electronic health records and survey answers in the All of Us Research Program","volume":"29","author":"Sulieman","year":"2022","journal-title":"J Am Med Inform Assoc."},{"key":"2025081903513118800_ocaf112-B18","first-page":"1044","article-title":"Calculating the benefits of a research patient data repository","volume":"2006","author":"Nalichowski","year":"2006","journal-title":"AMIA Annu Symp Proc"},{"key":"2025081903513118800_ocaf112-B19","doi-asserted-by":"crossref","first-page":"e202764","DOI":"10.1001\/jamanetworkopen.2020.2764","article-title":"Factors associated with increased collection of patient-reported outcomes within a large health care system","volume":"3","author":"Sisodia","year":"2020","journal-title":"JAMA Netw Open"},{"key":"2025081903513118800_ocaf112-B20","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1046\/j.1525-1497.2001.016009606.x","article-title":"The PHQ-9: validity of a brief depression severity measure","volume":"16","author":"Kroenke","year":"2001","journal-title":"J Gen Intern Med"},{"key":"2025081903513118800_ocaf112-B21","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.1001\/archinte.166.10.1092","article-title":"A brief measure for assessing generalized anxiety disorder: the GAD-7","volume":"166","author":"Spitzer","year":"2006","journal-title":"Arch Intern Med"},{"key":"2025081903513118800_ocaf112-B22","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1093\/jamia\/ocad247","article-title":"OHDSI standardized vocabularies\u2014a large-scale centralized reference ontology for international data harmonization","volume":"31","author":"Reich","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2025081903513118800_ocaf112-B23","doi-asserted-by":"crossref","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The Unified Medical Language System (UMLS): integrating biomedical terminology","volume":"32(Database Issue)","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2025081903513118800_ocaf112-B24","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"2025081903513118800_ocaf112-B25","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-82673-4","volume-title":"Applying Quantitative Bias Analysis to Epidemiologic Data","author":"Fox","year":"2021"},{"key":"2025081903513118800_ocaf112-B26","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1093\/jamia\/ocy173","article-title":"Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review","volume":"26","author":"Koleck","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2025081903513118800_ocaf112-B27","doi-asserted-by":"crossref","first-page":"e0286297","DOI":"10.1371\/journal.pone.0286297","article-title":"Researching COVID to Enhance Recovery (RECOVER) adult study protocol: rationale, objectives, and design","volume":"18","author":"Horwitz","year":"2023","journal-title":"PLoS One"},{"key":"2025081903513118800_ocaf112-B28","doi-asserted-by":"publisher","DOI":"10.1056\/cat.23.0404","article-title":"Ambient artificial intelligence scribes to alleviate the burden of clinical documentation","volume":"5","author":"Tierney","year":"2024","journal-title":"NEJM Catal Innov Care Deliv"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/9\/1454\/63796097\/ocaf112.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/9\/1454\/63796097\/ocaf112.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T07:51:38Z","timestamp":1755589898000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/9\/1454\/8206315"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,18]]},"references-count":28,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,7,18]]},"published-print":{"date-parts":[[2025,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaf112","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"type":"print","value":"1067-5027"},{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,7,18]]}}}