{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T14:17:10Z","timestamp":1793888230979,"version":"4.8.0"},"reference-count":17,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T00:00:00Z","timestamp":1711411200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"UCSF Bakar Computational Health Science Institute"},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["UL1TR001872"],"award-info":[{"award-number":["UL1TR001872"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["TL1TR001871"],"award-info":[{"award-number":["TL1TR001871"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["K99LM014099"],"award-info":[{"award-number":["K99LM014099"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,3,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P\u2005=\u2005.65) or area deprivation index (P\u2005=\u2005.80).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/ibd\/izae068","type":"journal-article","created":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T09:16:54Z","timestamp":1709803014000},"page":"665-670","source":"Crossref","is-referenced-by-count":7,"title":["Accurate, Robust, and Scalable Machine Abstraction of Mayo Endoscopic Subscores From Colonoscopy Reports"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1139-0900","authenticated-orcid":false,"given":"Anna L","family":"Silverman","sequence":"first","affiliation":[{"name":"Division of Gastroenterology and Hepatology, Department of Medicine, Mayo Clinic , Phoenix, AZ ,","place":["USA"]},{"name":"Department of Medicine, University of California, San Diego , La Jolla, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Balu","family":"Bhasuran","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco , San Francisco, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arman","family":"Mosenia","sequence":"additional","affiliation":[{"name":"UCSF School of Medicine, University of California, San Francisco , San Francisco, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fatema","family":"Yasini","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Berkeley , Berkeley, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gokul","family":"Ramasamy","sequence":"additional","affiliation":[{"name":"Department of Radiology, Mayo Clinic , Phoenix, AZ ,","place":["USA"]},{"name":"School of Computing and Augmented Intelligence, Arizona State University , Tempe, AZ ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Imon","family":"Banerjee","sequence":"additional","affiliation":[{"name":"Department of Radiology, Mayo Clinic , Phoenix, AZ ,","place":["USA"]},{"name":"School of Computing and Augmented Intelligence, Arizona State University , Tempe, AZ ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saransh","family":"Gupta","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Berkeley , Berkeley, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taline","family":"Mardirossian","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Berkeley , Berkeley, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rohan","family":"Narain","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Berkeley , Berkeley, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Sewell","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology, Department of Medicine, Zuckerberg San Francisco General Hospital , San Francisco, CA ,","place":["USA"]},{"name":"Division of Gastroenterology and Hepatology, Department of Medicine, University of California, San Francisco , San Francisco, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7433-2740","authenticated-orcid":false,"given":"Atul J","family":"Butte","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco , San Francisco, CA ,","place":["USA"]},{"name":"Center for Data-Driven Insights and Innovation, University of California Health , Oakland, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1789-3004","authenticated-orcid":false,"given":"Vivek A","family":"Rudrapatna","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco , San Francisco, CA ,","place":["USA"]},{"name":"Division of Gastroenterology and Hepatology, Department of Medicine, University of California, San Francisco , San Francisco, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,3,26]]},"reference":[{"issue":"5","key":"2025030423095544900_CIT0001","doi-asserted-by":"crossref","first-page":"1570","DOI":"10.1053\/j.gastro.2020.12.031","article-title":"STRIDE-II: an update on the Selecting Therapeutic Targets in Inflammatory Bowel Disease (STRIDE) initiative of the International Organization for the Study of IBD (IOIBD): determining therapeutic goals for treat-to-target strategies in IBD","volume":"160","author":"Turner","year":"2021","journal-title":"Gastroenterology."},{"key":"2025030423095544900_CIT0002","year":"2016"},{"issue":"5","key":"2025030423095544900_CIT0003","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc."},{"issue":"85","key":"2025030423095544900_CIT0004","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"2025030423095544900_CIT0005","article-title":"Autogluon-tabular: robust and accurate automl for structured data","author":"Erickson","year":"2020"},{"key":"2025030423095544900_CIT0006","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018"},{"issue":"4","key":"2025030423095544900_CIT0007","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1093\/ibd\/izac109","article-title":"Identifying the presence, activity, and status of extraintestinal manifestations of inflammatory bowel disease using natural language processing of clinical notes","volume":"29","author":"Stidham","year":"2023","journal-title":"Inflamm Bowel Dis."},{"issue":"7","key":"2025030423095544900_CIT0008","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1097\/MIB.0b013e31828133fd","article-title":"Improving case definition of Crohn\u2019s disease and ulcerative colitis in electronic medical records using natural language processing: a novel informatics approach","volume":"19","author":"Ananthakrishnan","year":"2013","journal-title":"Inflamm Bowel Dis."},{"issue":"10","key":"2025030423095544900_CIT0009","doi-asserted-by":"crossref","first-page":"2242","DOI":"10.1093\/ibd\/izy127","article-title":"The association between arthralgia and vedolizumab using natural language processing","volume":"24","author":"Cai","year":"2018","journal-title":"Inflamm Bowel Dis."},{"issue":"4","key":"2025030423095544900_CIT0010","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1097\/MEG.0000000000002317","article-title":"Clinical characteristics and prognostic factors for Crohn\u2019s disease relapses using natural language processing and machine learning: a pilot study","volume":"34","author":"Gomoll\u00f3n","year":"2022","journal-title":"Eur J Gastroenterol Hepatol"},{"issue":"4","key":"2025030423095544900_CIT0011","doi-asserted-by":"crossref","first-page":"936","DOI":"10.1007\/s10620-012-2433-8","article-title":"Automated identification of surveillance colonoscopy in inflammatory bowel disease using natural language processing","volume":"58","author":"Hou","year":"2013","journal-title":"Dig Dis Sci."},{"key":"2025030423095544900_CIT0012","first-page":"36","article-title":"Application of natural language processing to VA electronic health records to identify phenotypic characteristics for clinical and research purposes","volume":"2008","author":"Gundlapalli","year":"2008","journal-title":"Summit Transl Bioinform"},{"issue":"2","key":"2025030423095544900_CIT0013","doi-asserted-by":"crossref","first-page":"e30345","DOI":"10.2196\/30345","article-title":"Evaluation of natural language processing for the identification of Crohn disease\u2013related variables in Spanish electronic health records: a validation study for the PREMONITION-CD Project","volume":"10","author":"Montoto","year":"2022","journal-title":"JMIR Med Inform"},{"issue":"3","key":"2025030423095544900_CIT0014","first-page":"1238","article-title":"Introduction of an area deprivation index measuring patient socioeconomic status in an integrated health system: implications for population health","volume":"4","author":"Knighton","year":"2016","journal-title":"EGEMS (Wash DC)"},{"issue":"5","key":"2025030423095544900_CIT0015","doi-asserted-by":"crossref","first-page":"e193963","DOI":"10.1001\/jamanetworkopen.2019.3963","article-title":"Performance of a deep learning model vs human reviewers in grading endoscopic disease severity of patients with ulcerative colitis","volume":"2","author":"Stidham","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"3","key":"2025030423095544900_CIT0016","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1016\/j.gie.2020.08.011","article-title":"Fully automated endoscopic disease activity assessment in ulcerative colitis","volume":"93","author":"Yao","year":"2021","journal-title":"Gastrointest Endosc."},{"key":"2025030423095544900_CIT0017","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2020.10986","article-title":"Biased models have biased explanations","author":"Jain","year":"2020"}],"container-title":["Inflammatory Bowel Diseases"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ibdjournal\/advance-article-pdf\/doi\/10.1093\/ibd\/izae068\/57098293\/izae068.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ibdjournal\/article-pdf\/31\/3\/665\/57098293\/izae068.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ibdjournal\/article-pdf\/31\/3\/665\/57098293\/izae068.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,4]],"date-time":"2025-03-04T18:10:52Z","timestamp":1741111852000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ibdjournal\/article\/31\/3\/665\/7635742"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,26]]},"references-count":17,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3,26]]},"published-print":{"date-parts":[[2025,3,3]]}},"URL":"https:\/\/doi.org\/10.1093\/ibd\/izae068","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.06.19.22276606","asserted-by":"object"}]},"ISSN":["1078-0998","1536-4844"],"issn-type":[{"value":"1078-0998","type":"print"},{"value":"1536-4844","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,3]]},"published":{"date-parts":[[2024,3,26]]}}}