{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T18:23:16Z","timestamp":1790792596950,"version":"4.1.0"},"reference-count":45,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2021,5,8]],"date-time":"2021-05-08T00:00:00Z","timestamp":1620432000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["ascpt.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Clin Pharma and Therapeutics"],"published-print":{"date-parts":[[2021,8]]},"abstract":"<jats:p>Adverse drug reaction (ADR) reporting is a major component of drug safety monitoring; its input will, however, only be optimized if systems can manage to deal with its tremendous flow of information, based primarily on unstructured text fields. The aim of this study was to develop an automated system allowing to code ADRs from patient reports. Our system was based on a knowledge base about drugs, enriched by supervised machine learning (ML) models trained on patients reporting data. To train our models, we selected all cases of ADRs reported by patients to a French Pharmacovigilance Centre through a national web\u2010portal between March 2017 and March 2019 (<jats:italic>n<\/jats:italic>\u00a0=\u00a02,058 reports). We tested both conventional ML models and deep\u2010learning models. We performed an external validation using a dataset constituted of a random sample of ADRs reported to the Marseille Pharmacovigilance Centre over the same period (<jats:italic>n<\/jats:italic>\u00a0=\u00a0187). Here, we show that regarding area under the curve (AUC) and F\u2010measure, the best model to identify ADRs was gradient boosting trees (LGBM), with an AUC of 0.93 (0.92\u20130.94) and F\u2010measure of 0.72 (0.68\u20130.75). This model was run for external validation showing an AUC of 0.91 and a F\u2010measure of 0.58. We evaluated an artificial intelligence pipeline that was found able to learn how to identify correctly ADRs from unstructured data. This result allowed us to start a new study using more data to further improve our performance and offer a tool that is useful in practice to efficiently manage drug safety information.<\/jats:p>","DOI":"10.1002\/cpt.2266","type":"journal-article","created":{"date-parts":[[2021,4,18]],"date-time":"2021-04-18T18:21:05Z","timestamp":1618770065000},"page":"392-400","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["Artificial Intelligence for Unstructured Healthcare Data: Application to Coding of Patient Reporting of Adverse Drug Reactions"],"prefix":"10.1111","volume":"110","author":[{"given":"Louis","family":"L\u00e9tinier","sequence":"first","affiliation":[{"name":"INSERM BPH, U1219 Team Pharmacoepidemiology Univ. Bordeaux Bordeaux France"},{"name":"CHU de Bordeaux Pole de Sant\u00e9 Publique Service de Pharmacologie M\u00e9dicale Centre de Pharmacovigilance de Bordeaux Bordeaux France"},{"name":"Synapse Medicine Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julien","family":"Jouganous","sequence":"additional","affiliation":[{"name":"Synapse Medicine Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mehdi","family":"Benkebil","sequence":"additional","affiliation":[{"name":"Surveillance Division Agence nationale de s\u00e9curit\u00e9 du m\u00e9dicament et des produits de sant\u00e9 (ANSM) Saint Denis France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alicia","family":"Bel\u2010L\u00e9toile","sequence":"additional","affiliation":[{"name":"Synapse Medicine Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cl\u00e9ment","family":"Goehrs","sequence":"additional","affiliation":[{"name":"Synapse Medicine Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Allison","family":"Singier","sequence":"additional","affiliation":[{"name":"INSERM BPH, U1219 Team Pharmacoepidemiology Univ. Bordeaux Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Franck","family":"Rouby","sequence":"additional","affiliation":[{"name":"CRPV Marseille Provence Corse Service Hospitalo\u2010Universitaire de Pharmacologie Clinique et Pharmacovigilance Assistance Publique H\u00f4pitaux de Marseille Marseille France"},{"name":"Institut des Neurosciences des Syst\u00e8mes INSERM 1106 Aix Marseille Universit\u00e9 Marseille France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cl\u00e9mence","family":"Lacroix","sequence":"additional","affiliation":[{"name":"CRPV Marseille Provence Corse Service Hospitalo\u2010Universitaire de Pharmacologie Clinique et Pharmacovigilance Assistance Publique H\u00f4pitaux de Marseille Marseille France"},{"name":"Institut des Neurosciences des Syst\u00e8mes INSERM 1106 Aix Marseille Universit\u00e9 Marseille France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ghada","family":"Miremont","sequence":"additional","affiliation":[{"name":"INSERM BPH, U1219 Team Pharmacoepidemiology Univ. Bordeaux Bordeaux France"},{"name":"CHU de Bordeaux Pole de Sant\u00e9 Publique Service de Pharmacologie M\u00e9dicale Centre de Pharmacovigilance de Bordeaux Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jo\u00eblle","family":"Micallef","sequence":"additional","affiliation":[{"name":"CRPV Marseille Provence Corse Service Hospitalo\u2010Universitaire de Pharmacologie Clinique et Pharmacovigilance Assistance Publique H\u00f4pitaux de Marseille Marseille France"},{"name":"Institut des Neurosciences des Syst\u00e8mes INSERM 1106 Aix Marseille Universit\u00e9 Marseille France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francesco","family":"Salvo","sequence":"additional","affiliation":[{"name":"INSERM BPH, U1219 Team Pharmacoepidemiology Univ. Bordeaux Bordeaux France"},{"name":"CHU de Bordeaux Pole de Sant\u00e9 Publique Service de Pharmacologie M\u00e9dicale Centre de Pharmacovigilance de Bordeaux Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antoine","family":"Pariente","sequence":"additional","affiliation":[{"name":"INSERM BPH, U1219 Team Pharmacoepidemiology Univ. Bordeaux Bordeaux France"},{"name":"CHU de Bordeaux Pole de Sant\u00e9 Publique Service de Pharmacologie M\u00e9dicale Centre de Pharmacovigilance de Bordeaux Bordeaux France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2021,5,8]]},"reference":[{"key":"e_1_2_13_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(14)61099-0"},{"key":"e_1_2_13_2_1","doi-asserted-by":"publisher","DOI":"10.1155\/2017\/7983473"},{"key":"e_1_2_13_3_1","doi-asserted-by":"publisher","DOI":"10.2471\/BLT.17.197426"},{"key":"e_1_2_13_4_1","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.3340"},{"key":"e_1_2_13_5_1","unstructured":"Cambridge dictionary<https:\/\/dictionary.cambridge.org\/fr\/dictionnaire\/anglais\/artificial\u2010intelligence>. Accessed April 9 2021."},{"key":"e_1_2_13_6_1","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/cix731"},{"key":"e_1_2_13_7_1","doi-asserted-by":"publisher","DOI":"10.3390\/info6040848"},{"key":"e_1_2_13_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/pds.3967"},{"key":"e_1_2_13_9_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.279.15.1200"},{"key":"e_1_2_13_10_1","unstructured":"Aspden P.Preventing medication errors: quality chasm series<http:\/\/psnet.ahrq.gov\/issue\/preventing\u2010medication\u2010errors\u2010quality\u2010chasm\u2010series> (2006). Accessed June 24 2020."},{"key":"e_1_2_13_11_1","doi-asserted-by":"publisher","DOI":"10.1080\/14740338.2018.1491547"},{"key":"e_1_2_13_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.12266"},{"key":"e_1_2_13_13_1","first-page":"247","article-title":"MedDRA and pharmacovigilance a complex and little\u2010evaluated tool","volume":"25","author":"Prescrire","year":"2016","journal-title":"Prescrire Int."},{"key":"e_1_2_13_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tips.2019.07.005"},{"key":"e_1_2_13_15_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.1255"},{"key":"e_1_2_13_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-6080(09)03160-2"},{"key":"e_1_2_13_17_1","unstructured":"Camelot: PDF Table Extraction for Humans \u2014 Camelot 0.7.3 documentation<https:\/\/camelot\u2010py.readthedocs.io\/en\/master> (2020). Accessed March 6 2020."},{"key":"e_1_2_13_18_1","unstructured":"Ramos J.Using TF\u2010IDF to determine word relevance in document queries. <https:\/\/www.semanticscholar.org\/paper\/Using\u2010TF\u2010IDF\u2010to\u2010Determine\u2010Word\u2010Relevance\u2010in\u2010Queries\u2010Ramos\/b3bf6373ff41a115197cb5b30e57830c16130c2c> (2003). Accessed March 6 2020."},{"key":"e_1_2_13_19_1","unstructured":"Wydmuch M. Jasinska K. Kuznetsov M. Busa\u2010Fekete R.&Dembczynski K.A no\u2010regret generalization of hierarchical softmax to extreme multi\u2010labelclassification. NIPS 2018. <http:\/\/arXiv.org\/abs\/1810.11671> (2018). Accessed March 6 2020."},{"key":"e_1_2_13_20_1","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-140-10-200405180-00009"},{"key":"e_1_2_13_21_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781420075779"},{"key":"e_1_2_13_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/uog.2791"},{"key":"e_1_2_13_23_1","doi-asserted-by":"crossref","unstructured":"Mohamed S.S.&Salama M.M.A.Computer\u2010aided diagnosis for prostate cancer using support vector machine.Medical Imaging 2005: Visualization Image\u2010Guided Procedures and Display. International Society for Optics and Photonics <https:\/\/www.spiedigitallibrary.org\/conference\u2010proceedings\u2010of\u2010spie\/5744\/0000\/Computer\u2010aided\u2010diagnosis\u2010for\u2010prostate\u2010cancer\u2010using\u2010support\u2010vector\u2010machine\/10.1117\/12.598800.short> (2005). Accessed March 6 2020.","DOI":"10.1117\/12.598800"},{"key":"e_1_2_13_24_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_13_25_1","unstructured":"Breiman L.Bias variance and arcing classifiers<citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.115.7931&rep=rep1&type=pdf> (1996)."},{"key":"e_1_2_13_26_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"e_1_2_13_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2008.04.033"},{"key":"e_1_2_13_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_13_29_1","first-page":"1","article-title":"LightGBM: a highly efficient gradient boosting decision tree","volume":"30","author":"Ke G.","year":"2017","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"e_1_2_13_30_1","unstructured":"Bojanowski P. Grave E. Joulin A.&Mikolov T.Enriching word vectors with subword. Information. arXiv:160704606 <http:\/\/arxiv.org\/abs\/1607.04606> (2017). Accessed March 12 2020."},{"key":"e_1_2_13_31_1","unstructured":"Joulin A. Grave E. Bojanowski P.&Mikolov T.Bag of tricks for efficient text classification. arXiv:160701759. <http:\/\/arxiv.org\/abs\/1607.01759> (2016). Accessed March 12 2020."},{"key":"e_1_2_13_32_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_2_13_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_2_13_34_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_13_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-017-9746-6"},{"key":"e_1_2_13_36_1","first-page":"1227","article-title":"Classification of the severity of adverse drugs reactions","volume":"270","author":"Chauvet R.","year":"2020","journal-title":"Studies Health Technol. Informat."},{"key":"e_1_2_13_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40264-019-00869-4"},{"key":"e_1_2_13_38_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0118432"},{"key":"e_1_2_13_39_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2018\/6275435","article-title":"A framework of rebalancing imbalanced healthcare data for rare events\u2019 classification: a case of look\u2010alike sound\u2010alike mix\u2010up incident detection","volume":"2018","author":"Zhao Y.","year":"2018","journal-title":"J. Healthcare Eng."},{"key":"e_1_2_13_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2015.04.005"},{"key":"e_1_2_13_41_1","unstructured":"Letinier L.et al.patent FR1661257 \u2010 Device and method for generating a database relating to drugs. <https:\/\/patents.google.com\/patent\/FR3059118A1\/en>(2016). Accessed June 25 2020."},{"key":"e_1_2_13_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2014.2325029"},{"key":"e_1_2_13_43_1","unstructured":"Collobert R. Weston J. Bottou L. Karlen M. Kavukcuoglu K.&Kuksa P.Natural language processing (almost) from Scratch. arXiv:11030398 <http:\/\/arxiv.org\/abs\/1103.0398> (2011). Accessed June 25 2020."},{"key":"e_1_2_13_44_1","unstructured":"Devlin J. Chang M.\u2010W. Lee K.&Toutanova T.Pre\u2010training of deep bidirectional transformers for language understanding. arXiv:181004805 <http:\/\/arxiv.org\/abs\/1810.04805> (2019). Accessed December 11 2020."},{"key":"e_1_2_13_45_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2019.16489"}],"container-title":["Clinical Pharmacology &amp; Therapeutics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpt.2266","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/cpt.2266","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/ascpt.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpt.2266","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,7]],"date-time":"2023-11-07T13:38:28Z","timestamp":1699364308000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascpt.onlinelibrary.wiley.com\/doi\/10.1002\/cpt.2266"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,8]]},"references-count":45,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,8]]}},"alternative-id":["10.1002\/cpt.2266"],"URL":"https:\/\/doi.org\/10.1002\/cpt.2266","archive":["Portico"],"relation":{},"ISSN":["0009-9236","1532-6535"],"issn-type":[{"value":"0009-9236","type":"print"},{"value":"1532-6535","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,8]]},"assertion":[{"value":"2020-12-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-22","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}