{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T03:00:56Z","timestamp":1775617256941,"version":"3.50.1"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T00:00:00Z","timestamp":1710201600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM009886"],"award-info":[{"award-number":["R01LM009886"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM014344"],"award-info":[{"award-number":["R01LM014344"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Center for Advancing Clinical and Translational Science","award":["UL1TR001873"],"award-info":[{"award-number":["UL1TR001873"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,4,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Extracting PICO (Populations, Interventions, Comparison, and Outcomes) entities is fundamental to evidence retrieval. We present a novel method, PICOX, to extract overlapping PICO entities.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>PICOX first identifies entities by assessing whether a word marks the beginning or conclusion of an entity. Then, it uses a multi-label classifier to assign one or more PICO labels to a span candidate. PICOX was evaluated using 1 of the best-performing baselines, EBM-NLP, and 3 more datasets, ie, PICO-Corpus and randomized controlled trial publications on Alzheimer\u2019s Disease (AD) or COVID-19, using entity-level precision, recall, and F1 scores.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>PICOX achieved superior precision, recall, and F1 scores across the board, with the micro F1 score improving from 45.05 to 50.87 (P \u226a.01). On the PICO-Corpus, PICOX obtained higher recall and F1 scores than the baseline and improved the micro recall score from 56.66 to 67.33. On the COVID-19 dataset, PICOX also outperformed the baseline and improved the micro F1 score from 77.10 to 80.32. On the AD dataset, PICOX demonstrated comparable F1 scores with higher precision when compared to the baseline.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>PICOX excels in identifying overlapping entities and consistently surpasses a leading baseline across multiple datasets. Ablation studies reveal that its data augmentation strategy effectively minimizes false positives and improves precision.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae065","type":"journal-article","created":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T20:59:15Z","timestamp":1710277155000},"page":"1163-1171","source":"Crossref","is-referenced-by-count":17,"title":["A span-based model for extracting overlapping PICO entities from randomized controlled trial publications"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-0077-3615","authenticated-orcid":false,"given":"Gongbo","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York, NY 10032, United States"}]},{"given":"Yiliang","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Population Health Sciences, Weill Cornell Medicine , New York, NY 10065, United States"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2413-5918","authenticated-orcid":false,"given":"Yan","family":"Hu","sequence":"additional","affiliation":[{"name":"McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston , Houston, TX 77030, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5274-4672","authenticated-orcid":false,"given":"Hua","family":"Xu","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, Yale School of Medicine , New Haven, CT 06510, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9624-0214","authenticated-orcid":false,"given":"Chunhua","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York, NY 10032, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9309-8331","authenticated-orcid":false,"given":"Yifan","family":"Peng","sequence":"additional","affiliation":[{"name":"Department of Population Health Sciences, Weill Cornell Medicine , New York, NY 10065, United States"}]}],"member":"286","published-online":{"date-parts":[[2024,3,12]]},"reference":[{"issue":"3","key":"2024041923353054400_ocae065-B1","doi-asserted-by":"crossref","first-page":"A12","DOI":"10.7326\/ACPJC-1995-123-3-A12","article-title":"The well-built clinical question: a key to evidence-based decisions","volume":"123","author":"Richardson","year":"1995","journal-title":"ACP J Club"},{"issue":"6","key":"2024041923353054400_ocae065-B2","doi-asserted-by":"crossref","first-page":"1022","DOI":"10.1093\/jamia\/ocad036","article-title":"EvidenceMap: a three-level knowledge representation for medical evidence computation and comprehension","volume":"30","author":"Kang","year":"2023","journal-title":"J Am Med Inform Assoc"},{"issue":"7","key":"2024041923353054400_ocae065-B3","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1038\/s41591-023-02366-9","article-title":"AI-generated text may have a role in evidence-based medicine","volume":"29","author":"Peng","year":"2023","journal-title":"Nat Med"},{"issue":"2","key":"2024041923353054400_ocae065-B4","doi-asserted-by":"crossref","first-page":"e012545","DOI":"10.1136\/bmjopen-2016-012545","article-title":"Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry","volume":"7","author":"Borah","year":"2017","journal-title":"BMJ Open"},{"key":"2024041923353054400_ocae065-B5","author":"McCallum","year":"2003"},{"key":"2024041923353054400_ocae065-B6","first-page":"363","author":"Finkel","year":"2005"},{"key":"2024041923353054400_ocae065-B7","first-page":"1064","author":"Ma","year":"2016"},{"key":"2024041923353054400_ocae065-B8","first-page":"3879","author":"Yang","year":"2018"},{"key":"2024041923353054400_ocae065-B9","first-page":"188","article-title":"Pretraining to recognize PICO elements from randomized controlled trial literature","volume":"264","author":"Kang","year":"2019","journal-title":"Stud Health Technol Inform"},{"key":"2024041923353054400_ocae065-B10","first-page":"2227","author":"Peters","year":"2018"},{"key":"2024041923353054400_ocae065-B11","first-page":"4171","author":"Devlin","year":"2019"},{"issue":"1","key":"2024041923353054400_ocae065-B12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2021","journal-title":"ACM Trans Comput Healthcare"},{"issue":"1","key":"2024041923353054400_ocae065-B13","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1186\/s13643-022-02074-4","article-title":"PICO entity extraction for preclinical animal literature","volume":"11","author":"Wang","year":"2022","journal-title":"Syst Rev"},{"key":"2024041923353054400_ocae065-B14","first-page":"3615","author":"Beltagy","year":"2019"},{"issue":"4","key":"2024041923353054400_ocae065-B15","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2024041923353054400_ocae065-B16","first-page":"143","author":"Kanakarajan","year":"2021"},{"key":"2024041923353054400_ocae065-B17","author":"Nye","year":"2018"},{"key":"2024041923353054400_ocae065-B18","first-page":"5784","author":"Wadden","year":"2019"},{"key":"2024041923353054400_ocae065-B19","author":"Luan","year":"2019"},{"issue":"05","key":"2024041923353054400_ocae065-B20","doi-asserted-by":"crossref","first-page":"8401","DOI":"10.1609\/aaai.v34i05.6358","article-title":"HAMNER: headword amplified multi-span distantly supervised method for domain specific named entity recognition","volume":"34","author":"Liu","year":"2020","journal-title":"AAAI"},{"issue":"05","key":"2024041923353054400_ocae065-B21","doi-asserted-by":"crossref","first-page":"9016","DOI":"10.1609\/aaai.v34i05.6434","article-title":"Boundary enhanced neural span classification for nested named entity recognition","volume":"34","author":"Tan","year":"2020","journal-title":"AAAI"},{"key":"2024041923353054400_ocae065-B22","first-page":"7183","author":"Fu","year":"2021"},{"key":"2024041923353054400_ocae065-B23","first-page":"4814","author":"Li","year":"2021"},{"key":"2024041923353054400_ocae065-B24","first-page":"892","author":"Wan","year":"2022"},{"key":"2024041923353054400_ocae065-B25","first-page":"7096","author":"Zhu","year":"2022"},{"key":"2024041923353054400_ocae065-B26","first-page":"80","author":"Golam Sohrab","year":"2021"},{"issue":"11","key":"2024041923353054400_ocae065-B27","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1093\/bioinformatics\/btaa993","article-title":"A span-graph neural model for overlapping entity relation extraction in biomedical texts","volume":"37","author":"Fei","year":"2021","journal-title":"Bioinformatics"},{"key":"2024041923353054400_ocae065-B28","first-page":"97","author":"Zaratiana","year":"2022"},{"key":"2024041923353054400_ocae065-B29","first-page":"1","author":"Zaratiana","year":"2022"},{"key":"2024041923353054400_ocae065-B30","first-page":"2782","author":"Shen","year":"2021"},{"key":"2024041923353054400_ocae065-B31","first-page":"26","author":"Mutinda","year":"2022"},{"key":"2024041923353054400_ocae065-B32","first-page":"414","author":"Zhang","year":"2018"},{"issue":"9","key":"2024041923353054400_ocae065-B33","doi-asserted-by":"crossref","DOI":"10.1093\/bioinformatics\/btad542","article-title":"Towards precise PICO extraction from abstracts of randomized controlled trials using a section-specific learning approach","volume":"39","author":"Hu","year":"2023","journal-title":"Bioinformatics"},{"key":"2024041923353054400_ocae065-B34","first-page":"606","author":"Hu","year":"2022"},{"issue":"16","key":"2024041923353054400_ocae065-B35","doi-asserted-by":"crossref","first-page":"14612","DOI":"10.1609\/aaai.v35i16.17717","article-title":"Document-level relation extraction with adaptive thresholding and localized context pooling","volume":"35","author":"Zhou","year":"2021","journal-title":"AAAI"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocae065\/57122532\/ocae065.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/5\/1163\/57286293\/ocae065.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/5\/1163\/57286293\/ocae065.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,19]],"date-time":"2024-04-19T23:36:07Z","timestamp":1713569767000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/5\/1163\/7627401"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,12]]},"references-count":35,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,3,12]]},"published-print":{"date-parts":[[2024,4,19]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae065","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,5,1]]},"published":{"date-parts":[[2024,3,12]]}}}