{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T06:49:54Z","timestamp":1776926994395,"version":"3.51.2"},"reference-count":29,"publisher":"Georg Thieme Verlag KG","issue":"03","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\n          Background\u2003The 21st Century Cures Act mandates the immediate, electronic release of health information to patients. However, in the case of adolescents, special consideration is required to ensure that confidentiality is maintained. The detection of confidential content in clinical notes may support operational efforts to preserve adolescent confidentiality while implementing information sharing.<\/jats:p><jats:p>\n          Objectives\u2003This study aimed to determine if a natural language processing (NLP) algorithm can identify confidential content in adolescent clinical progress notes.<\/jats:p><jats:p>\n          Methods\u2003A total of 1,200 outpatient adolescent progress notes written between 2016 and 2019 were manually annotated to identify confidential content. Labeled sentences from this corpus were featurized and used to train a two-part logistic regression model, which provides both sentence-level and note-level probability estimates that a given text contains confidential content. This model was prospectively validated on a set of 240 progress notes written in May 2022. It was subsequently deployed in a pilot intervention to augment an ongoing operational effort to identify confidential content in progress notes. Note-level probability estimates were used to triage notes for review and sentence-level probability estimates were used to highlight high-risk portions of those notes to aid the manual reviewer.<\/jats:p><jats:p>\n          Results\u2003The prevalence of notes containing confidential content was 21% (255\/1,200) and 22% (53\/240) in the train\/test and validation cohorts, respectively. The ensemble logistic regression model achieved an area under the receiver operating characteristic of 90 and 88% in the test and validation cohorts, respectively. Its use in a pilot intervention identified outlier documentation practices and demonstrated efficiency gains over completely manual note review.<\/jats:p><jats:p>\n          Conclusion\u2003An NLP algorithm can identify confidential content in progress notes with high accuracy. Its human-in-the-loop deployment in clinical operations augmented an ongoing operational effort to identify confidential content in adolescent progress notes. These findings suggest NLP may be used to support efforts to preserve adolescent confidentiality in the wake of the information blocking mandate.<\/jats:p>","DOI":"10.1055\/a-2051-9764","type":"journal-article","created":{"date-parts":[[2023,3,11]],"date-time":"2023-03-11T00:01:51Z","timestamp":1678492911000},"page":"400-407","source":"Crossref","is-referenced-by-count":6,"title":["A Natural Language Processing Model to Identify Confidential Content in Adolescent Clinical Notes"],"prefix":"10.1055","volume":"14","author":[{"given":"Naveed","family":"Rabbani","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States"}]},{"given":"Michael","family":"Bedgood","sequence":"additional","affiliation":[{"name":"California Department of Public Health, Richmond, California, United States"}]},{"given":"Conner","family":"Brown","sequence":"additional","affiliation":[{"name":"Information Services Department, Lucile Packard Children's Hospital, Palo Alto, California, United States"}]},{"given":"Ethan","family":"Steinberg","sequence":"additional","affiliation":[{"name":"Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, California, United States"},{"name":"Department of Computer Science, Stanford University, Stanford, California, United States"}]},{"given":"Rachel L.","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Division of Adolescent Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States"}]},{"given":"Jennifer L.","family":"Carlson","sequence":"additional","affiliation":[{"name":"Division of Adolescent Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States"}]},{"given":"Natalie","family":"Pageler","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States"}]},{"given":"Keith E.","family":"Morse","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States"}]}],"member":"194","published-online":{"date-parts":[[2023,5,24]]},"reference":[{"issue":"12","key":"ref2","first-page":"e395","article-title":"US hospital engagement in core domains of interoperability","volume":"22","author":"A J Holmgren","year":"2016","journal-title":"Am J Manag Care"},{"issue":"02","key":"ref3","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1136\/amiajnl-2014-003144","article-title":"VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes","volume":"22","author":"K M Nazi","year":"2015","journal-title":"J Am Med Inform Assoc"},{"issue":"01","key":"ref4","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1055\/s-0038-1676588","article-title":"Qualitative and quantitative analysis of patients' perceptions of the patient portal experience with OpenNotes","volume":"10","author":"V K Mishra","year":"2019","journal-title":"Appl Clin Inform"},{"issue":"07","key":"ref5","doi-asserted-by":"crossref","first-page":"461","DOI":"10.7326\/0003-4819-157-7-201210020-00002","article-title":"Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead","volume":"157","author":"T Delbanco","year":"2012","journal-title":"Ann Intern Med"},{"issue":"05","key":"ref6","doi-asserted-by":"crossref","first-page":"e13876","DOI":"10.2196\/13876","article-title":"OpenNotes after 7 years: patient experiences with ongoing access to their clinicians' outpatient visit notes","volume":"21","author":"J Walker","year":"2019","journal-title":"J Med Internet Res"},{"issue":"10","key":"ref7","doi-asserted-by":"crossref","first-page":"e226","DOI":"10.2196\/jmir.4872","article-title":"Sharing physician notes through an electronic portal is associated with improved medication adherence: quasi-experimental study","volume":"17","author":"E Wright","year":"2015","journal-title":"J Med Internet Res"},{"issue":"03","key":"ref8","doi-asserted-by":"crossref","first-page":"e2020034199","DOI":"10.1542\/peds.2020-034199","article-title":"Implications of the 21st Century Cures Act in pediatrics","volume":"147","author":"N M Pageler","year":"2021","journal-title":"Pediatrics"},{"issue":"02","key":"ref9","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1016\/j.jadohealth.2020.10.020","article-title":"NASPAG\/SAHM Statement: the 21st Century Cures Act and adolescent confidentiality","volume":"68","author":"J Carlson","year":"2021","journal-title":"J Adolesc Health"},{"issue":"04","key":"ref10","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.pedhc.2021.03.005","article-title":"The 21st Century Cures Act and challenges to adolescent confidentiality","volume":"35","author":"N A Schapiro","year":"2021","journal-title":"J Pediatr Health Care"},{"issue":"06","key":"ref11","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1001\/jama.288.6.710","article-title":"Effect of mandatory parental notification on adolescent girls' use of sexual health care services","volume":"288","author":"D M Reddy","year":"2002","journal-title":"JAMA"},{"issue":"04","key":"ref12","first-page":"667","article-title":"Minors' rights to consent to treatment: navigating the complexity of State laws","volume":"37","author":"D M Vukadinovich","year":"2004","journal-title":"J Health Law"},{"issue":"01","key":"ref13","doi-asserted-by":"crossref","first-page":"46","DOI":"10.17294\/2330-0698.1656","article-title":"Confidential care for adolescents in the U.S. health care system","volume":"6","author":"P R Pathak","year":"2019","journal-title":"J Patient Cent Res Rev"},{"issue":"03","key":"ref14","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1016\/j.jadohealth.2019.03.021","article-title":"Confidentiality matters but how do we improve implementation in adolescent sexual and reproductive health care?","volume":"65","author":"S Pampati","year":"2019","journal-title":"J Adolesc Health"},{"issue":"06","key":"ref15","doi-asserted-by":"crossref","first-page":"e2021053458","DOI":"10.1542\/peds.2021-053458","article-title":"State-by-state variability in adolescent privacy laws","volume":"149","author":"M Sharko","year":"2022","journal-title":"Pediatrics"},{"issue":"24","key":"ref16","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1001\/jama.1995.03520480033036","article-title":"Adolescents' perceptions of factors affecting their decisions to seek health care","volume":"273","author":"K R Ginsburg","year":"1995","journal-title":"JAMA"},{"issue":"12","key":"ref17","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1001\/jama.1997.03550120089044","article-title":"Influence of physician confidentiality assurances on adolescents' willingness to disclose information and seek future health care. A randomized controlled trial","volume":"278","author":"C A Ford","year":"1997","journal-title":"JAMA"},{"issue":"05","key":"ref18","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/S1054-139X(03)00184-8","article-title":"Truth and consequences: ethics, confidentiality, and disclosure in adolescent longitudinal prevention research","volume":"33","author":"C Lothen-Kline","year":"2003","journal-title":"J Adolesc Health"},{"issue":"02","key":"ref19","doi-asserted-by":"crossref","first-page":"e34085","DOI":"10.2196\/34085","article-title":"The 21st Century Cures Act and multiuser electronic health record access: potential pitfalls of information release","volume":"24","author":"S Arvisais-Anhalt","year":"2022","journal-title":"J Med Internet Res"},{"issue":"11","key":"ref20","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1093\/jamia\/ocaa202","article-title":"Preserving privacy for pediatric patients and families: use of confidential note types in pediatric ambulatory care","volume":"27","author":"C R Parsons","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"07","key":"ref21","doi-asserted-by":"crossref","first-page":"e2219692","DOI":"10.1001\/jamanetworkopen.2022.19692","article-title":"Organizational perspectives on technical capabilities and barriers related to pediatric data sharing and confidentiality","volume":"5","author":"M Bedgood","year":"2022","journal-title":"JAMA Netw Open"},{"issue":"09","key":"ref25","doi-asserted-by":"crossref","first-page":"e2124733","DOI":"10.1001\/jamanetworkopen.2021.24733","article-title":"Assessment of Prevalence of Adolescent Patient Portal Account Access by Guardians","volume":"4","author":"W Ip","year":"2021","journal-title":"JAMA Netw Open"},{"issue":"06","key":"ref26","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1016\/j.jadohealth.2021.09.009","article-title":"Ensuring adolescent patient portal confidentiality in the Age of the Cures Act final rule","volume":"69","author":"J Xie","year":"2021","journal-title":"J Adolesc Health"},{"issue":"06","key":"ref27","doi-asserted-by":"crossref","first-page":"e38482","DOI":"10.2196\/38482","article-title":"Prevalence of sensitive terms in clinical notes using natural language processing techniques: observational study","volume":"10","author":"J Lee","year":"2022","journal-title":"JMIR Med Inform"},{"issue":"10","key":"ref28","doi-asserted-by":"crossref","first-page":"2116","DOI":"10.1093\/jamia\/ocab116","article-title":"Automated detection of substance use information from electronic health records for a pediatric population","volume":"28","author":"Y Ni","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"01","key":"ref29","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1055\/s-0041-1741483","article-title":"Lessons learned from OpenNotes learning mode and subsequent implementation across a pediatric health system","volume":"13","author":"A Murugan","year":"2022","journal-title":"Appl Clin Inform"},{"issue":"01","key":"ref31","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1177\/1833358319851305","article-title":"Computer-assisted clinical coding: a narrative review of the literature on its benefits, limitations, implementation and impact on clinical coding professionals","volume":"49","author":"S Campbell","year":"2020","journal-title":"HIM J"},{"key":"ref32","doi-asserted-by":"crossref","first-page":"S100","DOI":"10.1038\/d41586-019-02871-3","article-title":"An AI boost for clinical trials","volume":"573","author":"M Woo","year":"2019","journal-title":"Nature"},{"issue":"12","key":"ref36","first-page":"4","article-title":"How to protect maternal health information in newborn's medical record","volume":"35","author":"J P Scibilia","year":"2014","journal-title":"AAP News"},{"issue":"06","key":"ref37","doi-asserted-by":"crossref","first-page":"e222656","DOI":"10.1001\/jamahealthforum.2022.2656","article-title":"Protecting the privacy of reproductive health information after the fall of Roe v Wade","volume":"3","author":"K Spector-Bagdady","year":"2022","journal-title":"JAMA Health Forum"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2051-9764.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,29]],"date-time":"2023-05-29T23:02:42Z","timestamp":1685401362000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2051-9764"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,10]]},"references-count":29,"journal-issue":{"issue":"03","published-online":{"date-parts":[[2023,5,24]]},"published-print":{"date-parts":[[2023,5]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2051-9764","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,3,10]]},"published":{"date-parts":[[2023,3,10]]}}}