{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T19:57:19Z","timestamp":1791575839948,"version":"4.3.4"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,3,12]],"date-time":"2025-03-12T00:00:00Z","timestamp":1741737600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0"},{"start":{"date-parts":[[2025,3,12]],"date-time":"2025-03-12T00:00:00Z","timestamp":1741737600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0"}],"funder":[{"DOI":"10.13039\/501100001734","name":"Copenhagen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001734","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Drug Saf"],"published-print":{"date-parts":[[2025,7]]},"DOI":"10.1007\/s40264-025-01531-y","type":"journal-article","created":{"date-parts":[[2025,3,12]],"date-time":"2025-03-12T17:30:43Z","timestamp":1741800643000},"page":"805-820","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Off-the-Shelf Large Language Models for Causality Assessment of Individual Case Safety Reports: A Proof-of-Concept with COVID-19 Vaccines"],"prefix":"10.1007","volume":"48","author":[{"given":"Andrea","family":"Abate","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elisa","family":"Poncato","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria Antonietta","family":"Barbieri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Greg","family":"Powell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Rossi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simay","family":"Peker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anders","family":"Hviid","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Bate","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maurizio","family":"Sessa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,3,12]]},"reference":[{"key":"1531_CR1","doi-asserted-by":"publisher","first-page":"102712","DOI":"10.1016\/j.jaut.2021.102712","volume":"123","author":"K Kragholm","year":"2021","unstructured":"Kragholm K, Sessa M, Mulvad T, Andersen MP, Collatz-Christensen H, Blomberg SN, et al. Thrombocytopenia after COVID-19 vaccination. J Autoimmun. 2021;123:102712.","journal-title":"J Autoimmun"},{"key":"1531_CR2","doi-asserted-by":"publisher","unstructured":"Shakibfar S, Nyberg F, Li H, Zhao J, Nordeng HME, Sandve GKF, Pavlovic M, Hajiebrahimi M, Andersen M, Sessa M.\nArtificial intelligence-driven prediction of COVID-19-related hospitalization and death: a systematic review. Front\nPublic Health. 2023;11:1183725. https:\/\/doi.org\/10.3389\/fpubh.2023.1183725. PMID: 37408750; PMCID: PMC10319067.","DOI":"10.3389\/fpubh.2023.1183725"},{"key":"1531_CR3","doi-asserted-by":"publisher","first-page":"1258840","DOI":"10.3389\/fpubh.2023.1258840","volume":"11","author":"S Shakibfar","year":"2023","unstructured":"Shakibfar S, Zhao J, Li H, Nordeng H, Lupattelli A, Pavlovic M, et al. Machine learning-driven development of a disease risk score for COVID-19 hospitalization and mortality: a Swedish and Norwegian register-based study. Front Public Health. 2023;11:1258840.","journal-title":"Front Public Health"},{"key":"1531_CR4","doi-asserted-by":"publisher","first-page":"1627","DOI":"10.1111\/bcp.16044","volume":"90","author":"D Moreno-Martos","year":"2024","unstructured":"Moreno-Martos D, Zhao J, Li H, Nyberg F, Bj\u00f8rndal LD, Hajiebrahimi M, et al. Impact of the early COVID-19 pandemic on adult mental health-related dispensed medications, hospitalizations and specialist outpatient visits in Norway and Sweden: Interrupted time series analysis. Br J Clin Pharmacol. 2024;90:1627\u201336.","journal-title":"Br J Clin Pharmacol"},{"key":"1531_CR5","doi-asserted-by":"publisher","first-page":"233","DOI":"10.4103\/2229-3485.120173","volume":"4","author":"RP Naidu","year":"2013","unstructured":"Naidu RP. Causality assessment: a brief insight into practices in pharmaceutical industry. Perspect Clin Res. 2013;4:233\u20136.","journal-title":"Perspect Clin Res"},{"key":"1531_CR6","doi-asserted-by":"publisher","first-page":"4398","DOI":"10.1016\/j.vaccine.2015.07.035","volume":"33","author":"TT Shimabukuro","year":"2015","unstructured":"Shimabukuro TT, Nguyen M, Martin D, DeStefano F. Safety monitoring in the vaccine adverse event reporting system (VAERS). Vaccine. 2015;33:4398\u2013405.","journal-title":"Vaccine"},{"key":"1531_CR7","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1007\/s40264-017-0507-4","volume":"40","author":"S Colilla","year":"2017","unstructured":"Colilla S, Tov EY, Zhang L, Kurzinger M-L, Tcherny-Lessenot S, Penfornis C, et al. Validation of new signal detection methods for web query log data compared to signal detection algorithms used with FAERS. Drug Saf. 2017;40:399\u2013408.","journal-title":"Drug Saf"},{"key":"1531_CR8","doi-asserted-by":"publisher","first-page":"204209862211189","DOI":"10.1177\/20420986221118972","volume":"13","author":"A Rudolph","year":"2022","unstructured":"Rudolph A, Mitchell J, Barrett J, Sk\u00f6ld H, Taavola H, Erlanson N, et al. Global safety monitoring of COVID-19 vaccines: how pharmacovigilance rose to the challenge. Ther Adv Drug Saf. 2022;13:20420986221118972.","journal-title":"Ther Adv Drug Saf."},{"key":"1531_CR9","doi-asserted-by":"publisher","first-page":"1451","DOI":"10.1080\/14740338.2021.1955101","volume":"20","author":"M Sessa","year":"2021","unstructured":"Sessa M, Kragholm K, Hviid A, Andersen M. Thromboembolic events in younger women exposed to Pfizer-BioNTech or Moderna COVID-19 vaccines. Expert Opin Drug Saf. 2021;20:1451\u20133.","journal-title":"Expert Opin Drug Saf"},{"key":"1531_CR10","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1016\/j.phrs.2017.07.005","volume":"123","author":"A Mascolo","year":"2017","unstructured":"Mascolo A, Scavone C, Sessa M, di Mauro G, Cimmaruta D, Orlando V, et al. Can causality assessment fulfill the new European definition of adverse drug reaction? A review of methods used in spontaneous reporting. Pharmacol Res. 2017;123:122\u20139.","journal-title":"Pharmacol Res"},{"key":"1531_CR11","unstructured":"Naveed H, Khan AU, Qiu S, Saqib M, Anwar S, Usman M, et al. A comprehensive overview of large language models. arXiv Prepr arXiv230706435. 2023;"},{"key":"1531_CR12","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2022-065687","volume":"13","author":"A Alami","year":"2023","unstructured":"Alami A, Krewski D, Farhat N, Mattison D, Wilson K, Gravel CA, et al. Risk of myocarditis and pericarditis in mRNA COVID-19-vaccinated and unvaccinated populations: a systematic review and meta-analysis. BMJ Open. 2023;13: e065687.","journal-title":"BMJ Open"},{"key":"1531_CR13","doi-asserted-by":"publisher","first-page":"1210007","DOI":"10.3389\/fcvm.2023.1210007","volume":"10","author":"M Pareek","year":"2023","unstructured":"Pareek M, Sessa P, Polverino P, Sessa F, Kragholm KH, Sessa M. Myocarditis and pericarditis in individuals exposed to the Ad26. COV2. S, BNT162b2 mRNA, or mRNA-1273 SARS-CoV-2 vaccines. Front Cardiovasc Med. 2023;10:1210007.","journal-title":"Front Cardiovasc Med."},{"key":"1531_CR14","doi-asserted-by":"publisher","first-page":"955","DOI":"10.3390\/diagnostics11060955","volume":"11","author":"C Pomara","year":"2021","unstructured":"Pomara C, Sessa F, Ciaccio M, Dieli F, Esposito M, Giammanco GM, et al. COVID-19 vaccine and death: causality algorithm according to the WHO eligibility diagnosis. Diagnostics. 2021;11:955.","journal-title":"Diagnostics"},{"key":"1531_CR15","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1007\/s40264-023-01381-6","volume":"47","author":"G Dong","year":"2024","unstructured":"Dong G, Bate A, Haguinet F, Westman G, D\u00fcrlich L, Hviid A, et al. Optimizing signal management in a vaccine adverse event reporting system: a proof-of-concept with COVID-19 vaccines using signs, symptoms, and natural language processing. Drug Saf. 2024;47:173\u201382.","journal-title":"Drug Saf"},{"key":"1531_CR16","doi-asserted-by":"publisher","unstructured":"Al-Azzawi F, Mahmoud I, Haguinet F, Bate A, Sessa M. Developing an Artificial Intelligence-Guided Signal Detection in\nthe Food and Drug Administration Adverse Event Reporting System (FAERS): A Proof-of-Concept Study Using\nGalcanezumab and Simulated Data. Drug Saf. 2023;46(8):743-751. https:\/\/doi.org\/10.1007\/s40264-023-01317-0. Epub\n2023 Jun 10. PMID: 37300636; PMCID: PMC10345058.","DOI":"10.1007\/s40264-023-01317-0"},{"key":"1531_CR17","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1145\/1354923.1354925","volume":"35","author":"V Kutinsky","year":"2007","unstructured":"Kutinsky V. Client-server HTTP communications made easy. ACM SIGAPL APL Quote Quad. 2007;35:15\u201320.","journal-title":"ACM SIGAPL APL Quote Quad."},{"key":"1531_CR18","unstructured":"White J, Fu Q, Hays S, Sandborn M, Olea C, Gilbert H, et al. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv Prepr arXiv230211382. 2023;"},{"key":"1531_CR19","doi-asserted-by":"publisher","first-page":"102720","DOI":"10.1016\/j.acalib.2023.102720","volume":"49","author":"LS Lo","year":"2023","unstructured":"Lo LS. The CLEAR path: A framework for enhancing information literacy through prompt engineering. J Acad Libr. 2023;49:102720.","journal-title":"J Acad Libr"},{"key":"1531_CR20","unstructured":"Venables WN, Smith DM. The R development core team. An Introd to R, Version. 2003;1."},{"key":"1531_CR21","volume-title":"Sentiment analysis and opinion mining","author":"B Liu","year":"2022","unstructured":"Liu B. Sentiment analysis and opinion mining. Springer Nature; 2022."},{"key":"1531_CR22","unstructured":"Ramos J. Using tf-idf to determine word relevance in document queries. In: Proc first Instr Conf Mach Learn. Citeseer; 2003. p. 29\u201348."},{"key":"1531_CR23","doi-asserted-by":"crossref","unstructured":"Hu M, Liu B. Mining and summarizing customer reviews. In: Proc tenth ACM SIGKDD Int Conf Knowl Discov data Min. 2004. p. 168\u201377.","DOI":"10.1145\/1014052.1014073"},{"key":"1531_CR24","doi-asserted-by":"crossref","unstructured":"Liu B, Hu M, Cheng J. Opinion observer: analyzing and comparing opinions on the web. In: Proc 14th Int Conf World Wide Web. 2005. p. 342\u201351.","DOI":"10.1145\/1060745.1060797"},{"key":"1531_CR25","doi-asserted-by":"crossref","unstructured":"Manning CD, Surdeanu M, Bauer J, Finkel JR, Bethard S, McClosky D. The Stanford CoreNLP natural language processing toolkit. In: Proc 52nd Annu Meet Assoc Comput Linguist Syst Demonstr. 2014. p. 55\u201360.","DOI":"10.3115\/v1\/P14-5010"},{"key":"1531_CR26","unstructured":"Nielsen F\u00c5. A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. arXiv Prepr arXiv11032903. 2011."},{"key":"1531_CR27","doi-asserted-by":"crossref","unstructured":"Islam MZ, Liu J, Li J, Liu L, Kang W. A semantics aware random forest for text classification. In: Proc 28th ACM Int Conf Inf Knowl Manag. 2019; p. 1061\u201370.","DOI":"10.1145\/3357384.3357891"},{"key":"1531_CR28","doi-asserted-by":"publisher","first-page":"1700127","DOI":"10.1002\/minf.201700127","volume":"37","author":"JB Brown","year":"2018","unstructured":"Brown JB. Classifiers and their metrics quantified. Mol Inform. 2018;37:1700127.","journal-title":"Mol Inform."},{"key":"1531_CR29","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1016\/j.engappai.2017.03.008","volume":"62","author":"A Sarda-Espinosa","year":"2017","unstructured":"Sarda-Espinosa A, Subbiah S, Bartz-Beielstein T. Conditional inference trees for knowledge extraction from motor health condition data. Eng Appl Artif Intell. 2017;62:26\u201337.","journal-title":"Eng Appl Artif Intell"},{"key":"1531_CR30","doi-asserted-by":"publisher","first-page":"31","DOI":"10.4103\/0976-500X.179361","volume":"7","author":"SJ Thaker","year":"2016","unstructured":"Thaker SJ, Sinha RS, Gogtay NJ, Thatte UM. Evaluation of inter-rater agreement between three causality assessment methods used in pharmacovigilance. J Pharmacol Pharmacother. 2016;7:31\u20133.","journal-title":"J Pharmacol Pharmacother"},{"key":"1531_CR31","doi-asserted-by":"publisher","first-page":"e2428276","DOI":"10.1001\/jamanetworkopen.2024.28276","volume":"7","author":"ME Matheny","year":"2024","unstructured":"Matheny ME, Yang J, Smith JC, Walsh CG, Al-Garadi MA, Davis SE, et al. Enhancing postmarketing surveillance of medical products with large language models. JAMA Netw Open [Internet]. 2024;7:e2428276. https:\/\/doi.org\/10.1001\/jamanetworkopen.2024.28276.","journal-title":"JAMA Netw Open [Internet]"},{"key":"1531_CR32","doi-asserted-by":"publisher","unstructured":"Painter JL, Mahaux O, Vanini M, Kara V, Roshan C, Karwowski M, et al. Enhancing drug safety documentation search capabilities with Large Language Models: A user-centric approach. Int Conf Comput Sci Comput Intell Proc. (CSCI), Las Vegas, NV, USA, 2023; p. 49-56. https:\/\/doi.org\/10.1109\/CSCI62032.2023.00015.","DOI":"10.1109\/CSCI62032.2023.00015"},{"key":"1531_CR33","doi-asserted-by":"publisher","unstructured":"Rane N, Choudhary S, Rane J. Gemini versus ChatGPT: applications, performance, architecture, capabilities, and implementation. Performance, Archit Capab Implement. J. Artif. Intell. 2024;5(1):69-93. https:\/\/doi.org\/10.2139\/ssrn.4723687.","DOI":"10.2139\/ssrn.4723687"}],"container-title":["Drug Safety"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40264-025-01531-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40264-025-01531-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40264-025-01531-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T08:54:53Z","timestamp":1750150493000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40264-025-01531-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,12]]},"references-count":33,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["1531"],"URL":"https:\/\/doi.org\/10.1007\/s40264-025-01531-y","relation":{},"ISSN":["0114-5916","1179-1942"],"issn-type":[{"value":"0114-5916","type":"print"},{"value":"1179-1942","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,12]]},"assertion":[{"value":"17 February 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 March 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"No ethics approval is required for studies using publicly available data from VAERS.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"The data sources are open access and the datasets generated and\/or analyzed during the current study are available from the corresponding author on reasonable request.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Data Availability"}},{"value":"Code is available from the corresponding author upon reasonable request.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code Availability"}},{"value":"Not applicable. VAERS spontaneous reports are anonymous and publicly available.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate."}},{"value":"Not applicable. VAERS spontaneous reports are anonymous and publicly available.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication."}},{"value":"MS and AB conceived and designed the study. AB, and MS designed and supervised the research and analyzed the data. MB, AA, EP, MAB, GP, AR, SP, AH, AB, and MS wrote the original draft and reviewed and edited. AA, EP, MAB, GP, AR, SP, AH, AB, and MS participated in the interpretation of data, and revised and approved the final article as submitted.","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Authors\u2019 Contributions"}},{"value":"No funding was received for this study.","order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Funding"}},{"value":"Greg Powell and Andrew Bate are both employees of GSK and hold stock and stock options. All the other authors have no conflicts of interest to declare. Andrew Bate is an Editorial Board member of Drug Safety. Andrew Bate was not involved in the selection of peer reviewers for the manuscript nor in any of the subsequent editorial decisions.","order":9,"name":"Ethics","group":{"name":"EthicsHeading","label":"Potential Conflict of Interest"}}]}}