{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T11:34:11Z","timestamp":1768736051078,"version":"3.49.0"},"reference-count":32,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2021,12,21]],"date-time":"2021-12-21T00:00:00Z","timestamp":1640044800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100000133","name":"Agency for Healthcare Research and Quality","doi-asserted-by":"publisher","award":["1R03HS026291\u201001"],"award-info":[{"award-number":["1R03HS026291\u201001"]}],"id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000038","name":"U.S. Food and Drug Administration","doi-asserted-by":"publisher","award":["U01FD005478"],"award-info":[{"award-number":["U01FD005478"]}],"id":[{"id":"10.13039\/100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Pharmacoepidemiology and Drug"],"published-print":{"date-parts":[[2022,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>To develop an annotation model to apply natural language processing (NLP) to device adverse event reports and implement the model to evaluate the most frequently experienced events among women reporting a sterilization device removal.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We included adverse event reports from the Manufacturer and User Facility Device Experience database from January 2005 to June 2018 related to device removal following hysteroscopic sterilization. We used an iterative process to develop an annotation model that extracts six categories of desired information and applied the annotation model to train an NLP algorithm. We assessed the model performance using positive predictive value (PPV, also known as precision), sensitivity (also known as recall), and F<jats:sub>1<\/jats:sub> score (a combined measure of PPV and sensitivity). Using extracted variables, we summarized the reporting source, the presence of prespecified and other patient and device events, additional sterilizations and other procedures performed, and time from implantation to removal.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The overall F<jats:sub>1<\/jats:sub> score was 91.5% for labeled items and 93.9% for distinct events after excluding duplicates. A total of 16\u2009535 reports of device removal were analyzed. The most frequently reported patient and device events were abdominal\/pelvic\/genital pain (<jats:italic>N<\/jats:italic>\u00a0=\u00a013\u2009166, 79.6%) and device dislocation\/migration (<jats:italic>N<\/jats:italic>\u00a0=\u00a03180, 19.2%), respectively. Of those reporting an additional sterilization procedure, the majority had a hysterectomy or salpingectomy (<jats:italic>N<\/jats:italic>\u00a0=\u00a07932). One\u2010fifth of the cases that had device removal timing specified reported a removal after 7\u2009years following implantation (<jats:italic>N<\/jats:italic>\u00a0=\u00a02444\/11\u2009293).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>We present a roadmap to develop an annotation model for NLP to analyze device adverse event reports. The extracted information is informative and complements findings from previous research using administrative data.<\/jats:p><\/jats:sec>","DOI":"10.1002\/pds.5402","type":"journal-article","created":{"date-parts":[[2021,12,17]],"date-time":"2021-12-17T16:46:55Z","timestamp":1639759615000},"page":"442-451","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Assessing adverse event reports of hysteroscopic sterilization device removal using natural language processing"],"prefix":"10.1002","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6671-3474","authenticated-orcid":false,"given":"Jialin","family":"Mao","sequence":"first","affiliation":[{"name":"Department of Population Health Sciences Weill Cornell Medicine  New York New York USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Art","family":"Sedrakyan","sequence":"additional","affiliation":[{"name":"Department of Population Health Sciences Weill Cornell Medicine  New York New York USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianyi","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Population Health Sciences Weill Cornell Medicine  New York New York USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maryam","family":"Guiahi","sequence":"additional","affiliation":[{"name":"Department of Obstetrics and Gynecology, Division of Family Planning University of Colorado Anschutz Medical Campus  Aurora Colorado USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Scott","family":"Chudnoff","sequence":"additional","affiliation":[{"name":"Department of Obstetrics and Gynecology Stamford Hospital  Stamford Connecticut USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Madris","family":"Kinard","sequence":"additional","affiliation":[{"name":"Device Events  York Pennsylvania USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephen B.","family":"Johnson","sequence":"additional","affiliation":[{"name":"Department of Population Health New York University Langone Health  New York New York USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2021,12,21]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1093\/humrep\/deg256"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmig.2015.04.017"},{"key":"e_1_2_9_4_1","unstructured":"U.S. Food and Drug Administration.Essure benefits and risks;2020. Accessed Jun 8 2020.https:\/\/www.fda.gov\/medical-devices\/essure-permanent-birth-control\/essure-benefits-and-risks"},{"key":"e_1_2_9_5_1","unstructured":"U.S. Food and Drug Administration.FDA activities related to essure;2020. Accessed June 8 2020.https:\/\/www.fda.gov\/medical-devices\/essure-permanent-birth-control\/fda-activities-related-essure"},{"key":"e_1_2_9_6_1","unstructured":"McGinleyL.Sales of Essure birth control implant to be halted by Bayer; U.S. last to sell controversial device. Accessed June 8 2020.https:\/\/www.washingtonpost.com\/news\/to-your-health\/wp\/2018\/07\/20\/sales-of-essure-birth-control-implant-halted-by-bayer-u-s-was-last-to-sell-controversial-device\/"},{"key":"e_1_2_9_7_1","unstructured":"U.S. Food and Drug Administration.Statement from FDA Commissioner Scott Gottlieb M.D. on new steps to strengthen the long\u2010term safety oversight of the Essure device following discontinuation of its U.S. Sales;2018. Accessed Jun 8 2020.https:\/\/www.fda.gov\/news-events\/press-announcements\/statement-fda-commissioner-scott-gottlieb-md-new-steps-strengthen-long-term-safety-oversight-essure"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h5162"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.1097\/AOG.0000000000003092"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajog.2017.07.011"},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.21269"},{"key":"e_1_2_9_12_1","unstructured":"U.S. Food and Drug Administration.MAUDE \u2010 Manufacturer and User Facility Device Experience;2018. Accessed January 4 2019.https:\/\/www.accessdata.fda.gov\/scripts\/cdrh\/cfdocs\/cfmaude\/search.cfm"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40290-015-0106-6"},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1097\/00006250-200312000-00012"},{"issue":"1","key":"e_1_2_9_15_1","first-page":"3912","article-title":"Device failures associated with patient injuries during robot\u2010assisted laparoscopic surgeries: a comprehensive review of FDA MAUDE database","volume":"15","author":"Andonian S","year":"2008","journal-title":"Can J Urol"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1747-0803.2007.00107.x"},{"issue":"1","key":"e_1_2_9_17_1","first-page":"46","article-title":"Presentation and management of major complications of midurethral slings: are complications under\u2010reported?","volume":"26","author":"Deng DY","year":"2007","journal-title":"NeurourolUrodyn"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jtcvs.2009.02.032"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.hrthm.2007.03.041"},{"key":"e_1_2_9_20_1","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M3028"},{"key":"e_1_2_9_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/clpt.2012.54"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40264-018-0762-z"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2011.1204"},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0151470"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ahj.2017.10.013"},{"key":"e_1_2_9_26_1","unstructured":"Device events;2019. Accessed June 10 2019.https:\/\/www.deviceevents.com\/"},{"key":"e_1_2_9_27_1","unstructured":"U.S. Food and Drug Administration.Post\u2010approval studies (PAS);2020. Accessed June 8 2020.https:\/\/www.accessdata.fda.gov\/scripts\/cdrh\/cfdocs\/cfpma\/pma_pas.cfm?t_id=531556&c_id=3675"},{"key":"e_1_2_9_28_1","unstructured":"International Medical Device Regulators Forum.Adverse event terminology;2020. Accessed June 11 2021.http:\/\/imdrf.org\/workitems\/wi-aet.asp"},{"key":"e_1_2_9_29_1","unstructured":"RimK.Mae2: Portable annotation tool for general natural language use. Paper presented at: Proceedings of 12th Joint ACL\u2010ISO Workshop on Interoperable Semantic Annotation;2016."},{"key":"e_1_2_9_30_1","unstructured":"HonnibalM.spaCy 2: Natural language understanding with Bloom embeddings convolutional neural networks and incremental parsing;2020.https:\/\/spacy.io\/"},{"key":"e_1_2_9_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jogoh.2020.101772"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.07.010"},{"key":"e_1_2_9_33_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocx039"}],"container-title":["Pharmacoepidemiology and Drug Safety"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pds.5402","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/pds.5402","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pds.5402","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,27]],"date-time":"2023-08-27T04:53:26Z","timestamp":1693112006000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pds.5402"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,21]]},"references-count":32,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,4]]}},"alternative-id":["10.1002\/pds.5402"],"URL":"https:\/\/doi.org\/10.1002\/pds.5402","archive":["Portico"],"relation":{},"ISSN":["1053-8569","1099-1557"],"issn-type":[{"value":"1053-8569","type":"print"},{"value":"1099-1557","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,21]]},"assertion":[{"value":"2021-03-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-13","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}