{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T04:20:10Z","timestamp":1772166010863,"version":"3.50.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,1,4]],"date-time":"2024-01-04T00:00:00Z","timestamp":1704326400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,4]],"date-time":"2024-01-04T00:00:00Z","timestamp":1704326400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100004325","name":"AstraZeneca","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004325","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Knowledge graphs are well-suited for modeling complex, unstructured, and multi-source data and facilitating their analysis. During the COVID-19 pandemic, adverse event data were integrated into a knowledge graph to support vaccine safety surveillance and nimbly respond to urgent health authority questions. Here, we provide details of this post-marketing safety system using public data sources. In addition to challenges with varied data representations, adverse event reporting on the COVID-19 vaccines generated an unprecedented volume of data; an order of magnitude larger than adverse events for all previous vaccines. The Patient Safety Knowledge Graph (PSKG) is a robust data store to accommodate the volume of adverse event data and harmonize primary surveillance data sources.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We designed a semantic model to represent key safety concepts. We built an extract-transform-load (ETL) data pipeline to parse and import primary public data sources; align key elements such as vaccine names; integrated the Medical Dictionary for Regulatory Activities (MedDRA); and applied quality metrics. PSKG is deployed in a Neo4J graph database, and made available via a web interface and Application Programming Interfaces (APIs).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We import and align adverse event data and vaccine exposure data from 250 countries on a weekly basis, producing a graph with 4,340,980 nodes and 30,544,475 edges as of July 1, 2022. PSKG is used for ad-hoc analyses and periodic reporting for several widely available COVID-19 vaccines. Analysis code using the knowledge graph is 80% shorter than an equivalent implementation written entirely in Python, and runs over 200 times faster.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Organizing safety data into a concise model of nodes, properties, and edge relationships has greatly simplified analysis code by removing complex parsing and transformation algorithms from individual analyses and instead managing these centrally. The adoption of the knowledge graph transformed how the team answers key scientific and medical questions. Whereas previously an analysis would involve aggregating and transforming primary datasets from scratch to answer a specific question, the team can now iterate easily and respond as quickly as requests evolve (e.g., \u201cProduce vaccine-X safety profile for adverse event-Y by country instead of age-range\u201d).<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12911-023-02409-8","type":"journal-article","created":{"date-parts":[[2024,1,4]],"date-time":"2024-01-04T10:05:43Z","timestamp":1704362743000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["A patient safety knowledge graph supporting vaccine product development"],"prefix":"10.1186","volume":"24","author":[{"given":"Andrew M.","family":"Simms","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anshul","family":"Kanakia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Muhammad","family":"Sipra","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bhaskar","family":"Dutta","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Noel","family":"Southall","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,1,4]]},"reference":[{"key":"2409_CR1","doi-asserted-by":"crossref","unstructured":"AstraZeneca. Two billion doses of AstraZeneca\u2019s COVID-19 vaccine supplied to countries across the world less than 12 months after first approval. AstraZeneca; 2021. https:\/\/www.astrazeneca.com\/media-centre\/press-releases\/2021\/two-billion-doses-of-astrazenecas-covid-19-vaccine-supplied-to-countries-across-the-world-less-than-12-months-after-first-approval.html. Accessed 2 July 2022.","DOI":"10.1007\/s40278-022-22875-6"},{"key":"2409_CR2","unstructured":"US Department of Health and Human Services. VAERS - Report an Adverse Event. 2022. https:\/\/vaers.hhs.gov\/reportevent.html.\u00a0Accessed 2 July\u00a02022."},{"key":"2409_CR3","unstructured":"European Medicines Agency. ADR reporting - patient guideline. 2022. https:\/\/www.ema.europa.eu\/en\/documents\/regulatory-procedural-guideline\/adverse-drug-reaction-adr-reporting-patient-guideline_en.pdf.\u00a0Accessed 2 July 2022."},{"key":"2409_CR4","unstructured":"US Department of Health and Human Services. Vaccine Adverse Event Reporting System (VAERS). 2022. https:\/\/vaers.hhs.gov\/index.html.\u00a0Accessed 2 July 2022"},{"key":"2409_CR5","unstructured":"US Department of Health and Human Services. VAERS - Guide to Interpreting VAERS Data. 2022. https:\/\/vaers.hhs.gov\/data\/dataguide.html.\u00a0Accessed 2 July 2022."},{"key":"2409_CR6","unstructured":"European Medicines Agency. EudraVigilance \u2014 European Medicines Agency. 2022. https:\/\/www.ema.europa.eu\/en\/human-regulatory\/research-development\/pharmacovigilance\/eudravigilance\/eudravigilance-electronic-reporting.\u00a0Accessed 2 July 2022."},{"key":"2409_CR7","unstructured":"European Medicines Agency. European database of suspected adverse drug reaction reports. 2022. https:\/\/www.adrreports.eu\/en\/index.html.\u00a0Accessed 2 July 2022."},{"issue":"2","key":"2409_CR8","doi-asserted-by":"publisher","first-page":"109","DOI":"10.2165\/00002018-199920020-00002","volume":"20","author":"E Brown","year":"1999","unstructured":"Brown E, Wood L, Wood S. The Medical Dictionary for Regulatory Activities (MedDRA). Drug Saf. 1999;20(2):109\u201317.","journal-title":"Drug Saf."},{"issue":"2","key":"2409_CR9","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/BF03256752","volume":"23","author":"P Mozzicato","year":"2009","unstructured":"Mozzicato P. MedDRA An Overview of the Medical Dictionary for Regulatory Activities. Pharm Med. 2009;23(2):65.","journal-title":"Pharm Med."},{"key":"2409_CR10","unstructured":"US Centers for Disease Control and Prevention. COVID Data Tracker. 2022. https:\/\/covid.cdc.gov\/covid-data-tracker\/#datatracker-home.\u00a0Accessed 2 July 2022."},{"key":"2409_CR11","unstructured":"US Centers for Disease Control and Prevention. COVID-19 Vaccinations in the United States, Jurisdiction. 2022. https:\/\/data.cdc.gov\/Vaccinations\/COVID-19-Vaccinations-in-the-United-States-Jurisdi\/unsk-b7fc.\u00a0Accessed 2 July 2022."},{"key":"2409_CR12","unstructured":"European Centre for Disease Prevention and Control. Homepage \u2014 European Centre for Disease Prevention and Control. 2022. https:\/\/www.ecdc.europa.eu\/en.\u00a0Accessed 2 July 2022."},{"key":"2409_CR13","unstructured":"European Centre for Disease Prevention and Control. Download COVID-19 data sets \u2014  European Centre for Disease Prevention and Control. 2022. https:\/\/www.ecdc.europa.eu\/en\/covid-19\/data. Accessed 2 July 2022."},{"key":"2409_CR14","doi-asserted-by":"crossref","unstructured":"Deutsch P. RFC1952: GZIP File Format Specification Version 4.3. USA: RFC Editor; 1996.","DOI":"10.17487\/rfc1952"},{"key":"2409_CR15","doi-asserted-by":"publisher","unstructured":"Shafranovich Y. Common Format and MIME Type for Comma-Separated Values (CSV) Files. RFC Editor; 2005. RFC 4180. https:\/\/doi.org\/10.17487\/RFC4180. https:\/\/www.rfc-editor.org\/info\/rfc4180.","DOI":"10.17487\/RFC4180"},{"key":"2409_CR16","unstructured":"European Medicines Agency. Signal management \u2014 European Medicines Agency. 2022. https:\/\/www.ema.europa.eu\/en\/human-regulatory\/post-authorisation\/pharmacovigilance\/signal-management#designated-medical-events-section.\u00a0Accessed 2 July\u00a02022."},{"key":"2409_CR17","unstructured":"European Medicines Agency. Important medical event terms list (MedDRA version 25.0). 2022. https:\/\/www.ema.europa.eu\/documents\/other\/meddra-important-medical-event-terms-list-version-250_en.xlsx.\u00a0Accessed 2 July 2022."},{"key":"2409_CR18","doi-asserted-by":"publisher","unstructured":"Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10(6):483\u20136. https:\/\/doi.org\/10.1002\/pds.677. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1002\/pds.677","DOI":"10.1002\/pds.677"},{"issue":"4","key":"2409_CR19","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1136\/amiajnl-2011-000116","volume":"18","author":"SJ Nelson","year":"2011","unstructured":"Nelson SJ, Zeng K, Kilbourne J, Powell T, Moore R. Normalized names for clinical drugs: RxNorm at 6 years. J Am Med Inform Assoc. 2011;18(4):441\u20138.","journal-title":"J Am Med Inform Assoc."},{"key":"2409_CR20","unstructured":"Neo4J, Inc . Graph Data Platform | Graph Database Management System | Neo4j. 2022. https:\/\/www.neo4j.com.\u00a0Accessed 30 May\u00a02022."},{"key":"2409_CR21","unstructured":"Neo4J, Inc . The Neo4j Cypher Manual v4.4. 2022. https:\/\/neo4j.com\/docs\/cypher-manual\/4.4\/.\u00a0Accessed 30 May\u00a02022."},{"key":"2409_CR22","unstructured":"R Core Team. Data Frames. 2022. https:\/\/www.rdocumentation.org\/packages\/base\/versions\/3.6.2\/topics\/data.frame.\u00a0Accessed 18 June 2022."},{"key":"2409_CR23","unstructured":"Pandas Development Team. Pandas DataFrame. 2022. https:\/\/pandas.pydata.org\/docs\/reference\/api\/pandas.DataFrame.html.\u00a0Accessed 18 June 2022."},{"key":"2409_CR24","unstructured":"Team PD. pandas documentation; pandas 1.4.3 documentation. 2022. https:\/\/pandas.pydata.org\/docs\/index.html.\u00a0Accessed 3 July\u00a02022."},{"issue":"3","key":"2409_CR25","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1561\/1900000024","volume":"5","author":"D Abadi","year":"2013","unstructured":"Abadi D, Boncz P, Ieos S, Harizopoulos S, Madden S. The Design and Implementation of Modern Column-Oriented Database Systems. Found Trends Databases. 2013;5(3):197\u2013280.","journal-title":"Found Trends Databases."},{"key":"2409_CR26","unstructured":"Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I. Spark: Cluster Computing with Working Sets. In: 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 10). Boston: USENIX Association; 2010. p. 1\u20137. https:\/\/www.usenix.org\/conference\/hotcloud-10\/spark-cluster-computing-working-sets."},{"issue":"4","key":"2409_CR27","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2020","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234\u201340.","journal-title":"Bioinformatics."},{"key":"2409_CR28","unstructured":"Huang K, Altosaar J, Ranganath R. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342. 2019."},{"key":"2409_CR29","doi-asserted-by":"crossref","unstructured":"Alsentzer E, Murphy JR, Boag W, Weng WH, Jin D, Naumann T, et\u00a0al. Publicly available clinical BERT embeddings. arXiv preprint arXiv:1904.03323. 2019.","DOI":"10.18653\/v1\/W19-1909"},{"key":"2409_CR30","doi-asserted-by":"crossref","unstructured":"Johnson AE, Pollard TJ, Shen L, Lehman LwH, Feng M, Ghassemi M, et\u00a0al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3(1):1\u20139.","DOI":"10.1038\/sdata.2016.35"},{"issue":"2","key":"2409_CR31","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1002\/wsbm.1323","volume":"8","author":"MR Boland","year":"2016","unstructured":"Boland MR, Jacunski A, Lorberbaum T, Romano JD, Moskovitch R, Tatonetti NP. Systems biology approaches for identifying adverse drug reactions and elucidating their underlying biological mechanisms. Wiley Interdiscip Rev Syst Biol Med. 2016;8(2):104\u201322.","journal-title":"Wiley Interdiscip Rev Syst Biol Med."},{"issue":"23","key":"2409_CR32","doi-asserted-by":"publisher","first-page":"3498","DOI":"10.2174\/1381612822666160509125047","volume":"22","author":"TB Ho","year":"2016","unstructured":"Ho TB, Le L, Thai DT, Taewijit S. Data-driven approach to detect and predict adverse drug reactions. Curr Pharm Des. 2016;22(23):3498\u2013526.","journal-title":"Curr Pharm Des."},{"issue":"1","key":"2409_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-017-16674-x","volume":"7","author":"DM Bean","year":"2017","unstructured":"Bean DM, Wu H, Iqbal E, Dzahini O, Ibrahim ZM, Broadbent M, et al. Knowledge graph prediction of unknown adverse drug reactions and validation in electronic health records. Sci Rep. 2017;7(1):1\u201311.","journal-title":"Sci Rep."},{"key":"2409_CR34","unstructured":"Yacoumatos C, Bragaglia S, Kanakia A, Svang\u00e5rd N, Mangion J, Donoghue C, et\u00a0al. TrialGraph: Machine Intelligence Enabled Insight from Graph Modelling of Clinical Trials. arXiv preprint arXiv:2112.08211. 2021."},{"issue":"3","key":"2409_CR35","doi-asserted-by":"publisher","first-page":"485","DOI":"10.3390\/sym13030485","volume":"13","author":"M Wang","year":"2021","unstructured":"Wang M, Qiu L, Wang X. A survey on knowledge graph embeddings for link prediction. Symmetry. 2021;13(3):485.","journal-title":"Symmetry."},{"key":"2409_CR36","doi-asserted-by":"crossref","unstructured":"Bhowmik R, Melo Gd. Explainable link prediction for emerging entities in knowledge graphs. In: International Semantic Web Conference. Springer; 2020. p. 39\u201355.","DOI":"10.1007\/978-3-030-62419-4_3"},{"key":"2409_CR37","doi-asserted-by":"crossref","unstructured":"Barbieri N, Bonchi F, Manco G. Who to follow and why: link prediction with explanations. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014. p. 1266\u20131275.","DOI":"10.1145\/2623330.2623733"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02409-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-023-02409-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02409-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,4]],"date-time":"2024-01-04T22:03:12Z","timestamp":1704405792000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-023-02409-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,4]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["2409"],"URL":"https:\/\/doi.org\/10.1186\/s12911-023-02409-8","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1918107\/v1","asserted-by":"object"}]},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,4]]},"assertion":[{"value":"1 August 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 December 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"AS was employed as a contractor at AstraZeneca, AK and NS are full-time employees of AstraZeneca, MS was a contractor at AstraZeneca, BD was formerly a full-time employee of AstraZeneca.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"10"}}