{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T16:57:17Z","timestamp":1794070637006,"version":"4.8.0"},"reference-count":56,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T00:00:00Z","timestamp":1658793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJMS"],"abstract":"<jats:p>Usefulness of Vaccine-Adverse Event-Reporting System (VAERS) data and protocols required for statistical analyses were pinpointed with a set of recommendations for the application of machine learning modeling or exploratory analyses on VAERS data with a case study of COVID-19 vaccines (Pfizer-BioNTech, Moderna, Janssen). A total of 262,454 duplicate reports (29%) from 905,976 reports were identified, which were merged into a total of 643,522 distinct reports. A customized online survey was also conducted providing 211 reports. A total of 20 highest reported adverse events were first identified. Differences in results after applying various machine learning algorithms (association rule mining, self-organizing maps, hierarchical clustering, bipartite graphs) on VAERS data were noticed. Moderna reports showed injection-site-related AEs of higher frequencies by 15.2%, consistent with the online survey (12% higher reporting rate for pain in the muscle for Moderna compared to Pfizer-BioNTech). AEs {headache, pyrexia, fatigue, chills, pain, dizziness} constituted &gt;50% of the total reports. Chest pain in male children reports was 295% higher than in female children reports. Penicillin and sulfa were of the highest frequencies (22%, and 19%, respectively). Analysis of uncleaned VAERS data demonstrated major differences from the above (7% variations). Spelling\/grammatical mistakes in allergies were discovered (e.g., ~14% reports with incorrect spellings for penicillin).<\/jats:p>","DOI":"10.3390\/ijms23158235","type":"journal-article","created":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T04:59:16Z","timestamp":1658897956000},"page":"8235","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Usefulness of Vaccine Adverse Event Reporting System for Machine-Learning Based Vaccine Research: A Case Study for COVID-19 Vaccines"],"prefix":"10.3390","volume":"23","author":[{"given":"James","family":"Flora","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, California State University San Bernardino, 5500 University Parkway, San Bernardino, CA 92407, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7511-3873","authenticated-orcid":false,"given":"Wasiq","family":"Khan","sequence":"additional","affiliation":[{"name":"School of Computer Science and Mathematics, Liverpool John Moores University, Liverpool L3 3AF, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer","family":"Jin","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, California State University San Bernardino, 5500 University Parkway, San Bernardino, CA 92407, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Jin","sequence":"additional","affiliation":[{"name":"Division of Vascular & Interventional Radiology, Department of Radiology, Loma Linda University Medical Center, Loma Linda, CA 92354, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8413-0045","authenticated-orcid":false,"given":"Abir","family":"Hussain","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, University of Sharjah, Sharjah P.O. Box 27272, United Arab Emirates"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khalil","family":"Dajani","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, California State University San Bernardino, 5500 University Parkway, San Bernardino, CA 92407, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8549-2099","authenticated-orcid":false,"given":"Bilal","family":"Khan","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, California State University San Bernardino, 5500 University Parkway, San Bernardino, CA 92407, USA"},{"name":"Institute of the Environment and Sustainability, University of California Los Angeles, Los Angeles, CA 90095, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,26]]},"reference":[{"key":"ref_1","unstructured":"U.S. Department of Health and Human Services (HHS) (2022, May 23). About VAERS-Background and Public Health Importance 2022, Available online: https:\/\/vaers.hhs.gov\/about.html."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4398","DOI":"10.1016\/j.vaccine.2015.07.035","article-title":"Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS)","volume":"33","author":"Shimabukuro","year":"2015","journal-title":"Vaccine"},{"key":"ref_3","unstructured":"Ball, R., Braun, M.M., Chen, R.T., Ellenberg, S.S., English-Bullard, R., Haber, P., and Zhou, W. (2003). Surveillance for safety after immunization: Vaccine Adverse Event Reporting System (VAERS)\u2014United States, 1991\u20132001. MWWR. Surveillance Summaries: Morbidity and Mortality Weekly Report. Surveillance Summaries, CDC."},{"key":"ref_4","unstructured":"Su, J.R. (2022, May 23). Myopericarditis Following COVID-19 Vaccination: Updates from the Vaccine Adverse Event Reporting System (VAERS) [Internet]. Team. CDCC-19 VTFVS, Editor. Atanta, GA; (ACIP Meeting COVID-19 Vaccines; Volume 202113), Available online: https:\/\/stacks.cdc.gov\/view\/cdc\/110920."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"6291","DOI":"10.1016\/j.vaccine.2020.07.039","article-title":"Adverse events following quadrivalent meningococcal diphtheria toxoid conjugate vaccine (Menactra\u00ae) reported to the Vaccine Adverse Event Reporting System (VAERS), 2005\u20132016","volume":"38","author":"Myers","year":"2020","journal-title":"Vaccine"},{"key":"ref_6","unstructured":"VAERS (2022, May 23). VAERS Data 2021, Available online: https:\/\/vaers.hhs.gov\/data\/datasets.html?."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7458","DOI":"10.1016\/j.vaccine.2020.09.072","article-title":"The reporting sensitivity of the Vaccine Adverse Event Reporting System (VAERS) for anaphylaxis and for Guillain-Barr\u00e9 syndrome","volume":"38","author":"Miller","year":"2020","journal-title":"Vaccine"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1136\/amiajnl-2010-000022","article-title":"Text mining for the Vaccine Adverse Event Reporting System: Medical text classification using informative feature selection","volume":"18","author":"Botsis","year":"2011","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1393","DOI":"10.1093\/jamia\/ocab014","article-title":"Extracting postmarketing adverse events from safety reports in the vaccine adverse event reporting system (VAERS) using deep learning","volume":"28","author":"Du","year":"2021","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Lian, A.T., Du, J., and Tang, L. (2022). Using a Machine Learning Approach to Monitor COVID-19 Vaccine Adverse Events (VAE) from Twitter Data. Vaccines, 10.","DOI":"10.3390\/vaccines10010103"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"525","DOI":"10.32604\/iasc.2022.021216","article-title":"Prediction of Suitable Candidates for COVID-19 Vaccination","volume":"32","author":"Sujatha","year":"2022","journal-title":"Intell. Autom. Soft Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"34318","DOI":"10.1038\/srep34318","article-title":"Statistical and Ontological Analysis of Adverse Events Associated with Monovalent and Combination Vaccines against Hepatitis A and B Diseases","volume":"6","author":"Xie","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1963","DOI":"10.1080\/21645515.2018.1456598","article-title":"Post-licensure safety surveillance of zoster vaccine live (Zostavax\u00ae) in the United States, Vaccine Adverse Event Reporting System (VAERS), 2006\u20132015","volume":"14","author":"Miller","year":"2018","journal-title":"Hum. Vaccin Immunother."},{"key":"ref_14","first-page":"602","article-title":"Prediction of post-vaccination Guillain-Barr\u00e9 syndrome using data from a passive surveillance system","volume":"30","author":"Luo","year":"2021","journal-title":"Pharm. Drug Saf."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1324","DOI":"10.1016\/j.toxrep.2021.06.020","article-title":"Vaccines and sudden infant death: An analysis of the VAERS database 1990\u20132019 and review of the medical literature","volume":"8","author":"Miller","year":"2021","journal-title":"Toxicol. Rep."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"864","DOI":"10.1093\/cid\/civ430","article-title":"Advanced Clinical Decision Support for Vaccine Adverse Event Detection and Reporting","volume":"61","author":"Baker","year":"2015","journal-title":"Clin. Infect. Dis."},{"key":"ref_17","first-page":"e58","article-title":"Adverse events following measles, mumps, and rubella vaccine in adults reported to the vaccine adverse event reporting system (VAERS), 2003\u20132013","volume":"60","author":"Sukumaran","year":"2015","journal-title":"Clin. Infect. Dis."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Moro, P.L., Woo, E.J., Paul, W., Lewis, P., Petersen, B.W., and Cano, M. (2016). Post-Marketing Surveillance of Human Rabies Diploid Cell Vaccine (Imovax) in the Vaccine Adverse Event Reporting System (VAERS) in the United States, 1990\u20132015. PLoS Negl. Trop. Dis., 10.","DOI":"10.1371\/journal.pntd.0004846"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7253","DOI":"10.1016\/j.vaccine.2012.09.074","article-title":"Causality assessment of adverse events reported to the Vaccine Adverse Event Reporting System (VAERS)","volume":"30","author":"Loughlin","year":"2012","journal-title":"Vaccine"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1758","DOI":"10.1016\/j.vaccine.2017.02.030","article-title":"Adverse events following quadrivalent meningococcal CRM-conjugate vaccine (Menveo\u00ae) reported to the Vaccine Adverse Event Reporting system (VAERS), 2010\u20132015","volume":"35","author":"Myers","year":"2017","journal-title":"Vaccine"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gatti, M., Raschi, E., Moretti, U., Ardizzoni, A., Poluzzi, E., and Diemberger, I. (2021). Influenza vaccination and myo-pericarditis in patients receiving immune checkpoint inhibitors: Investigating the likelihood of interaction through the vaccine adverse event reporting system and vigibase. Vaccines, 9.","DOI":"10.3390\/vaccines9010019"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1007\/s40264-017-0523-4","article-title":"Using Probabilistic Record Linkage of Structured and Unstructured Data to Identify Duplicate Cases in Spontaneous Adverse Event Reporting Systems","volume":"40","author":"Kreimeyer","year":"2017","journal-title":"Drug Saf."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1038\/s41541-019-0132-6","article-title":"The how\u2019s and what\u2019s of vaccine reactogenicity","volume":"4","author":"Didierlaurent","year":"2019","journal-title":"npj Vaccines"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1423","DOI":"10.1016\/j.jaip.2020.12.047","article-title":"mRNA Vaccines to Prevent COVID-19 Disease and Reported Allergic Reactions: Current Evidence and Suggested Approach","volume":"9","author":"Banerji","year":"2021","journal-title":"J. Allergy Clin. Immunol Pract."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Croall, I.D., Trott, N., Rej, A., Aziz, I., O\u2019Brien, D.J., George, H.A., and Sanders, D.S. (2019). A Population Survey of Dietary Attitudes towards Gluten. Nutrients, 11, Available online: http:\/\/europepmc.org\/abstract\/MED\/31195638.","DOI":"10.3390\/nu11061276"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1097\/MCG.0b013e3181c41d12","article-title":"Fatigue as a Determinant of Health in Patients With Celiac Disease","volume":"44","author":"Vivancos","year":"2010","journal-title":"J. Clin. Gastroenterol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"9233","DOI":"10.3748\/wjg.v21.i31.9233","article-title":"Iron deficiency anemia in celiac disease","volume":"21","author":"Freeman","year":"2015","journal-title":"World J. Gastroenterol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1970","DOI":"10.1016\/j.procs.2014.05.181","article-title":"Mining Association Rules from Gene Ontology and Protein Networks: Promises and Challenges","volume":"29","author":"Guzzi","year":"2014","journal-title":"Procedia Comput. Sci."},{"key":"ref_29","unstructured":"Schuchat, A., Anderson, L.J., Rodewald, L.E., Cox, N.J., Hajjeh, R., Pallansch, M.A., Wharton, M., and National Center for Immunization and Respiratory Diseases (NCIRD), Division of Viral Diseases (2022, May 23). V-Safe After Health Checker, Available online: https:\/\/www.cdc.gov\/coronavirus\/2019-ncov\/vaccines\/safety\/vsafe.html."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1093\/bioinformatics\/btu260","article-title":"Using association rule mining to determine promising secondary phenotyping hypotheses","volume":"30","author":"Oellrich","year":"2014","journal-title":"Bioinformatics"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1093\/bib\/bbt074","article-title":"A primer to frequent itemset mining for bioinformatics","volume":"16","author":"Naulaerts","year":"2015","journal-title":"Brief. Bioinform."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Park, S.H., Reyes, J.A., Gilbert, D.R., Kim, J.W., and Kim, S. (2009). Prediction of protein-protein interaction types using association rule based classification. BMC Bioinform., 10.","DOI":"10.1186\/1471-2105-10-36"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Nafar, Z., and Golshani, A. (2006, January 7\u201310). Data Mining Methods for Protein-Protein Interactions. Proceedings of the 2006 Canadian Conference on Electrical and Computer Engineering, Ottawa, ON, Canada.","DOI":"10.1109\/CCECE.2006.277746"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1039\/C3AN01409F","article-title":"Association rule mining of cellular responses induced by metal and metal oxide nanoparticles","volume":"139","author":"Liu","year":"2014","journal-title":"Analytical"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mallik, S., Mukhopadhyay, A., Maulik, U., and Bandyopadhyay, S. (2013, January 16\u201319). Integrated analysis of gene expression and genome-wide DNA methylation for tumor prediction: An association rule mining-based approach. Proceedings of the 2013 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), Singapore.","DOI":"10.1109\/CIBCB.2013.6595397"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2643","DOI":"10.1093\/bioinformatics\/btn490","article-title":"GenMiner: Mining non-redundant association rules from integrated gene expression data and annotations","volume":"24","author":"Martinez","year":"2008","journal-title":"Bioinformatics"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1093\/bib\/bbp042","article-title":"Gene association analysis: A survey of frequent pattern mining from gene expression data","volume":"1","author":"Alves","year":"2010","journal-title":"Brief. Bioinform."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.ecoinf.2010.11.002","article-title":"Self-Organizing Maps applied to ecological sciences","volume":"6","author":"Chon","year":"2011","journal-title":"Ecol. Inform."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2907","DOI":"10.1073\/pnas.96.6.2907","article-title":"Interpreting patterns of gene expression with self-organizing maps: Methods and application to hematopoietic differentiation","volume":"96","author":"Tamayo","year":"1999","journal-title":"Proc. Natl. Acad Sci. USA."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1016\/S0014-5793(99)00524-4","article-title":"Analysis of gene expression data using self-organizing maps","volume":"451","author":"Kolehmainen","year":"1999","journal-title":"FEBS Lett."},{"key":"ref_41","unstructured":"Bullinaria, J.A. (2017, June 03). Self Organizing Maps: Fundamentals 2004. Available online: http:\/\/www.cs.bham.ac.uk\/~jxb\/NN\/l16.pdf."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1093\/gji\/ggu280","article-title":"Trans-dimensional finite-fault inversion","volume":"199","author":"Dettmer","year":"2014","journal-title":"Geophys. J. Int."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1002\/aic.10116","article-title":"Estimation of infinite dilution activity coefficients of organic compounds in water with neural classifiers","volume":"50","author":"Giralt","year":"2004","journal-title":"AIChE J."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, R., Lin, S., Rallo, R., Zhao, Y., Damoiseaux, R., Xia, T., and Cohen, Y. (2012). Automated Phenotype Recognition for Zebrafish Embryo Based In Vivo High Throughput Toxicity Screening of Engineered Nanomaterials. PLoS ONE, 7.","DOI":"10.1371\/journal.pone.0035014"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1695","DOI":"10.1021\/es103606x","article-title":"Self-Organizing Map Analysis of Toxicity-Related Cell Signaling Pathways for Metal and Metal Oxide Nanoparticles","volume":"45","author":"Rallo","year":"2011","journal-title":"Environ. Sci. Technol."},{"key":"ref_46","unstructured":"Rhodes, B.C., Mahaffey, J.A., and Cannady, J.D. (2000, January 16\u201319). Multiple self-organizing maps for intrusion detection. Proceedings of the 23rd National Information Systems Security Conference, Baltimore, MD, USA."},{"key":"ref_47","unstructured":"Greenacre, M., and Primicerio, R. (2013). Multivariate Analysis of Ecological Data, Fundaci\u00f3n BBVA."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v014.i15","article-title":"Arules\u2014A Computational Environment for Mining Association Rules and Frequent Item Sets","volume":"14","author":"Hahsler","year":"2005","journal-title":"J. Stat. Softw."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v087.i07","article-title":"Flexible Self-Organizing Maps in kohonen 3.0","volume":"87","author":"Wehrens","year":"2018","journal-title":"J. Stat. Softw."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Kokoska, S., and Zwillinger, D. (2000). CRC Standard Probability and Statistics Tables and Formulae, CRC Press. [1st ed.].","DOI":"10.1201\/b16923"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1007\/s00357-014-9161-z","article-title":"Ward\u2019s Hierarchical Agglomerative Clustering Method: Which Algorithms Implement Ward\u2019s Criterion?","volume":"31","author":"Murtagh","year":"2014","journal-title":"J. Classif."},{"key":"ref_52","unstructured":"(2022, May 23). Pasha. biPartite Graphs\u2014bl.ocks.org. Available online: http:\/\/bl.ocks.org\/NPashaP\/3ba0031d3d555afca4713e5264455025."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1111\/2041-210X.12139","article-title":"A method for detecting modules in quantitative bipartite networks","volume":"5","author":"Dormann","year":"2014","journal-title":"Methods Ecol. Evol."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"7","DOI":"10.2174\/1874213000902010007","article-title":"Indices, Graphs and Null Models: Analyzing Bipartite Ecological Networks","volume":"2","author":"Dormann","year":"2009","journal-title":"Open Ecol. J."},{"key":"ref_55","first-page":"2413793","article-title":"Introducing the bipartite package: Analysing ecological networks","volume":"1","author":"Dormann","year":"2008","journal-title":"Interaction"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Zhang, C., and Zhang, S. (2002). Association Rule Mining: Models and Algorithms, Springer.","DOI":"10.1007\/3-540-46027-6"}],"container-title":["International Journal of Molecular Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1422-0067\/23\/15\/8235\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:56:49Z","timestamp":1760140609000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1422-0067\/23\/15\/8235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,26]]},"references-count":56,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["ijms23158235"],"URL":"https:\/\/doi.org\/10.3390\/ijms23158235","relation":{},"ISSN":["1422-0067"],"issn-type":[{"value":"1422-0067","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,26]]}}}