{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T00:24:38Z","timestamp":1784939078859,"version":"3.55.0"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T00:00:00Z","timestamp":1669248000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T00:00:00Z","timestamp":1669248000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the Ministry of Food and Drug Safety of South Korea","award":["20200509312-00"],"award-info":[{"award-number":["20200509312-00"]}]},{"name":"the Ministry of Food and Drug Safety of South Korea","award":["21153MFDS607"],"award-info":[{"award-number":["21153MFDS607"]}]},{"name":"the Ministry of Trade, Industry & Energy","award":["No. 20015086"],"award-info":[{"award-number":["No. 20015086"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Safety profiles of the influenza vaccine and its subtypes are still limited. We aimed to address this knowledge gap using multiple data mining methods and calculated performance measurements to evaluate the precision of different detection methods. We conducted a post-marketing surveillance study between 2005 and 2019 using the Korea Adverse Event Reporting System database. Three data mining methods were applied: (a) proportional reporting ratio, (b) information component, and (c) tree-based scan statistics. We evaluated the performance of each method in comparison with the known adverse events (AEs) described in the labeling information. Compared to other vaccines, we identified 36 safety signals for the influenza vaccine, and 7 safety signals were unlabeled. In subtype-stratified analyses, application site disorders were reported more frequently with quadrivalent and cell-based vaccines, while a wide range of AEs were noted for trivalent and egg-based vaccines. Tree-based scan statistics showed well-balanced performance. Among the detected signals of influenza vaccines, narcolepsy requires special attention. A wider range of AEs were detected as signals for trivalent and egg-based vaccines. Although tree-based scan statistics showed balanced performance, complementary use of other techniques would be beneficial when large noise due to false positives is expected.<\/jats:p>","DOI":"10.1038\/s41598-022-21986-8","type":"journal-article","created":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T07:42:05Z","timestamp":1669362125000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Post-marketing surveillance study on influenza vaccine in South Korea using a nationwide spontaneous reporting database with multiple data mining methods"],"prefix":"10.1038","volume":"12","author":[{"given":"Hyesung","family":"Lee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Hong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"SangHee","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ju Hwan","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nam-Kyong","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2032-112X","authenticated-orcid":false,"given":"Sun-Young","family":"Jung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1010-7525","authenticated-orcid":false,"given":"Ju-Young","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,11,24]]},"reference":[{"key":"21986_CR1","doi-asserted-by":"publisher","first-page":"1285","DOI":"10.1016\/S0140-6736(17)33293-2","volume":"391","author":"AD Iuliano","year":"2018","unstructured":"Iuliano, A. D. et al. Estimates of global seasonal influenza-associated respiratory mortality: A modelling study. Lancet 391, 1285\u20131300 (2018).","journal-title":"Lancet"},{"issue":"11","key":"21986_CR2","doi-asserted-by":"publisher","first-page":"e1001558","DOI":"10.1371\/journal.pmed.1001558","volume":"10","author":"L Simonsen","year":"2013","unstructured":"Simonsen, L. et al. Global mortality estimates for the 2009 Influenza pandemic from the GLaMOR project: A modeling study. PLoS Med. 10(11), e1001558 (2013).","journal-title":"PLoS Med."},{"issue":"1","key":"21986_CR3","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1016\/S1473-3099(11)70295-X","volume":"12","author":"MT Osterholm","year":"2012","unstructured":"Osterholm, M. T., Kelley, N. S., Sommer, A. & Belongia, E. A. Efficacy and effectiveness of influenza vaccines: A systematic review and meta-analysis. Lancet Infect. Dis. 12(1), 36\u201344 (2012).","journal-title":"Lancet Infect. Dis."},{"key":"21986_CR4","doi-asserted-by":"publisher","first-page":"623","DOI":"10.3349\/ymj.2020.61.7.623","volume":"61","author":"D Yoon","year":"2020","unstructured":"Yoon, D., Kim, J. H., Lee, H. & Shin, J. Y. Updates on vaccine safety and post-licensure surveillance for adverse events following immunization in South Korea, 2005\u20132017. Yonsei Med. J. 61, 623\u2013630 (2020).","journal-title":"Yonsei Med. J."},{"key":"21986_CR5","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h3521","volume":"351","author":"J Aronson","year":"2015","unstructured":"Aronson, J. Safety. BMJ 351, h3521 (2015).","journal-title":"BMJ"},{"key":"21986_CR6","doi-asserted-by":"publisher","first-page":"1255","DOI":"10.1016\/S0140-6736(00)02799-9","volume":"356","author":"IR Edwards","year":"2000","unstructured":"Edwards, I. R. & Aronson, J. K. Adverse drug reactions: Definitions, diagnosis, and management. Lancet 356, 1255\u20131259 (2000).","journal-title":"Lancet"},{"key":"21986_CR7","doi-asserted-by":"publisher","first-page":"1815","DOI":"10.1016\/j.vaccine.2015.02.035","volume":"33","author":"C Kim","year":"2015","unstructured":"Kim, C. et al. Pandemic influenza A vaccination and incidence of Guillain-Barr\u00e9 syndrome in Korea. Vaccine 33, 1815\u20131823 (2015).","journal-title":"Vaccine"},{"key":"21986_CR8","doi-asserted-by":"publisher","first-page":"99","DOI":"10.7883\/yoken.65.99","volume":"65","author":"JH Kim","year":"2012","unstructured":"Kim, J. H. et al. Adverse events following immunization (AEFI) with the novel influenza a (H1N1) 2009 vaccine: findings from the national registry of all vaccine recipients and AEFI and the passive surveillance system in South Korea. Jpn. J. Infect. Dis. 65, 99\u2013104 (2012).","journal-title":"Jpn. J. Infect. Dis."},{"key":"21986_CR9","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.1016\/j.vaccine.2011.12.042","volume":"30","author":"Z Leroy","year":"2012","unstructured":"Leroy, Z., Broder, K., Menschik, D., Shimabukuro, T. & Martin, D. Febrile seizures after 2010\u20132011 influenza vaccine in young children, United States: a vaccine safety signal from the vaccine adverse event reporting system. Vaccine 30, 2020\u20132023 (2012).","journal-title":"Vaccine"},{"key":"21986_CR10","doi-asserted-by":"publisher","first-page":"896","DOI":"10.1056\/NEJMoa030595","volume":"350","author":"M Mutsch","year":"2004","unstructured":"Mutsch, M. et al. Use of the inactivated intranasal influenza vaccine and the risk of Bell\u2019s palsy in Switzerland. N. Engl. J. Med. 350, 896\u2013903 (2004).","journal-title":"N. Engl. J. Med."},{"key":"21986_CR11","doi-asserted-by":"publisher","first-page":"e33536","DOI":"10.1371\/journal.pone.0033536","volume":"7","author":"H Nohynek","year":"2012","unstructured":"Nohynek, H. et al. AS03 adjuvanted AH1N1 vaccine associated with an abrupt increase in the incidence of childhood narcolepsy in Finland. PLoS ONE 7, e33536 (2012).","journal-title":"PLoS ONE"},{"key":"21986_CR12","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1002\/pds.998","volume":"13","author":"W Zhou","year":"2004","unstructured":"Zhou, W. et al. A potential signal of Bell\u2019s palsy after parenteral inactivated influenza vaccines: Reports to the vaccine adverse event reporting system (VAERS)\u2013United States, 1991\u20132001. Pharmacoepidemiol. Drug Saf. 13, 505\u2013510 (2004).","journal-title":"Pharmacoepidemiol. Drug Saf."},{"key":"21986_CR13","doi-asserted-by":"publisher","first-page":"2272","DOI":"10.1016\/j.vaccine.2011.01.039","volume":"29","author":"R Baxter","year":"2011","unstructured":"Baxter, R. et al. Evaluation of the safety, reactogenicity and immunogenicity of FluBlok\u00ae trivalent recombinant baculovirus-expressed hemagglutinin influenza vaccine administered intramuscularly to healthy adults 50\u201364 years of age. Vaccine 29, 2272\u20132278 (2011).","journal-title":"Vaccine"},{"key":"21986_CR14","doi-asserted-by":"publisher","first-page":"6589","DOI":"10.1016\/j.vaccine.2009.08.032","volume":"27","author":"JC King Jr","year":"2009","unstructured":"King, J. C. Jr. et al. Evaluation of the safety, reactogenicity and immunogenicity of FluBlok trivalent recombinant baculovirus-expressed hemagglutinin influenza vaccine administered intramuscularly to healthy children aged 6\u201359 months. Vaccine 27, 6589\u20136594 (2009).","journal-title":"Vaccine"},{"key":"21986_CR15","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1086\/605505","volume":"200","author":"A Szymczakiewicz-Multanowska","year":"2009","unstructured":"Szymczakiewicz-Multanowska, A. et al. Safety and immunogenicity of a novel influenza subunit vaccine produced in mammalian cell culture. J. Infect. Dis. 200, 841\u2013848 (2009).","journal-title":"J. Infect. Dis."},{"key":"21986_CR16","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1093\/jamia\/ocv063","volume":"23","author":"HJ Duggirala","year":"2016","unstructured":"Duggirala, H. J. et al. Use of data mining at the food and drug administration. J. Am. Med. Inform. Assoc. 23, 428\u2013434 (2016).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"21986_CR17","doi-asserted-by":"publisher","first-page":"3702","DOI":"10.1016\/j.vaccine.2020.04.007","volume":"38","author":"JH Kim","year":"2020","unstructured":"Kim, J. H., Lee, H. & Shin, J. Y. Bacillus calmette-guerin (BCG) vaccine safety surveillance in the Korea adverse event reporting system using the tree-based scan statistic and conventional disproportionality-based algorithms. Vaccine 38, 3702\u20133710 (2020).","journal-title":"Vaccine"},{"key":"21986_CR18","doi-asserted-by":"publisher","first-page":"242","DOI":"10.3390\/vaccines8020242","volume":"8","author":"H Lee","year":"2020","unstructured":"Lee, H., Kim, J. H., Choe, Y. J. & Shin, J. Y. Safety surveillance of pneumococcal vaccine using three algorithms: Disproportionality methods, empirical bayes geometric mean, and tree-based scan statistic. Vaccines 8, 242 (2020).","journal-title":"Vaccines"},{"key":"21986_CR19","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1111\/1541-0420.00039","volume":"59","author":"M Kulldorff","year":"2003","unstructured":"Kulldorff, M., Fang, Z. & Walsh, S. J. A tree-based scan statistic for database disease surveillance. Biometrics 59, 323\u2013331 (2003).","journal-title":"Biometrics"},{"key":"21986_CR20","doi-asserted-by":"publisher","first-page":"883","DOI":"10.1007\/s40261-020-00952-0","volume":"40","author":"A Kamath","year":"2020","unstructured":"Kamath, A., Maity, N. & Nayak, M. A. Facial paralysis following influenza vaccination: A disproportionality analysis using the vaccine adverse event reporting system database. Clin. Drug. Investig. 40, 883\u2013889 (2020).","journal-title":"Clin. Drug. Investig."},{"key":"21986_CR21","doi-asserted-by":"publisher","first-page":"5464","DOI":"10.1016\/j.vaccine.2020.06.038","volume":"38","author":"H Lee","year":"2020","unstructured":"Lee, H., Kim, H. J., Choe, Y. J. & Shin, J. Y. Signals and trends of Guillain-Barr\u00e9 syndrome after the introduction of live-attenuated vaccines for influenza in the US and South Korean adverse event reporting systems. Vaccine 38, 5464\u20135473 (2020).","journal-title":"Vaccine"},{"key":"21986_CR22","doi-asserted-by":"publisher","first-page":"294ra105","DOI":"10.1126\/scitranslmed.aab2354","volume":"7","author":"SS Ahmed","year":"2015","unstructured":"Ahmed, S. S. et al. Antibodies to influenza nucleoprotein cross-react with human hypocretin receptor 2. Sci. Transl. Med. 7, 294ra105 (2015).","journal-title":"Sci. Transl. Med."},{"key":"21986_CR23","doi-asserted-by":"publisher","first-page":"4092","DOI":"10.1016\/j.vaccine.2016.06.064","volume":"34","author":"AM Moa","year":"2016","unstructured":"Moa, A. M., Chughtai, A. A., Muscatello, D. J., Turner, R. M. & MacIntyre, C. R. Immunogenicity and safety of inactivated quadrivalent influenza vaccine in adults: A systematic review and meta-analysis of randomised controlled trials. Vaccine 34, 4092\u20134102 (2016).","journal-title":"Vaccine"},{"key":"21986_CR24","doi-asserted-by":"publisher","first-page":"1856","DOI":"10.1016\/j.vaccine.2017.02.066","volume":"35","author":"JT Treanor","year":"2017","unstructured":"Treanor, J. T. et al. Immunogenicity and safety of a quadrivalent inactivated influenza vaccine compared with two trivalent inactivated influenza vaccines containing alternate B strains in adults: A phase 3, randomized noninferiority study. Vaccine 35, 1856\u20131864 (2017).","journal-title":"Vaccine"},{"key":"21986_CR25","doi-asserted-by":"publisher","first-page":"871","DOI":"10.1001\/jamainternmed.2017.0911","volume":"177","author":"C Carroll","year":"2017","unstructured":"Carroll, C. & Hassanin, A. Polypharmacy in the elderly-when good drugs lead to bad outcomes: A teachable moment. JAMA Intern. Med. 177, 871 (2017).","journal-title":"JAMA Intern. Med."},{"key":"21986_CR26","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.ijid.2016.06.018","volume":"49","author":"J Diez-Domingo","year":"2016","unstructured":"Diez-Domingo, J. et al. Safety and tolerability of cell culture-derived and egg-derived trivalent influenza vaccines in 3 to <18-year-old children and adolescents at risk of influenza-related complications. Int. J. Infect. Dis. 49, 171\u2013178 (2016).","journal-title":"Int. J. Infect. Dis."},{"key":"21986_CR27","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1097\/INF.0000000000001973","volume":"37","author":"CE Oh","year":"2018","unstructured":"Oh, C. E. et al. A randomized, double-blind, active-controlled clinical trial of a cell culture-derived inactivated trivalent influenza vaccine (NBP607) in healthy children 6 months through 18 years of age. Pediatr. Infect. Dis. J. 37, 605\u2013611 (2018).","journal-title":"Pediatr. Infect. Dis. J."},{"key":"21986_CR28","doi-asserted-by":"publisher","first-page":"178","DOI":"10.4161\/hv.6.2.9899","volume":"6","author":"A Pandey","year":"2010","unstructured":"Pandey, A., Singh, N., Sambhara, S. & Mittal, S. K. Egg-independent vaccine strategies for highly pathogenic H5N1 influenza viruses. Hum. Vaccin. 6, 178\u2013188 (2010).","journal-title":"Hum. Vaccin."},{"key":"21986_CR29","doi-asserted-by":"publisher","first-page":"1874","DOI":"10.1080\/21645515.2018.1460297","volume":"14","author":"A P\u00e9rez Rubio","year":"2018","unstructured":"P\u00e9rez Rubio, A. & Eiros, J. M. Cell culture-derived flu vaccine present and future. Hum. Vaccin. Immunother. 14, 1874\u20131882 (2018).","journal-title":"Hum. Vaccin. Immunother."},{"key":"21986_CR30","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1002\/pds.3423","volume":"22","author":"M Kulldorff","year":"2013","unstructured":"Kulldorff, M. et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol. Drug Saf. 22, 517\u2013523 (2013).","journal-title":"Pharmacoepidemiol. Drug Saf."},{"key":"21986_CR31","doi-asserted-by":"publisher","first-page":"1706","DOI":"10.2105\/AJPH.85.12.1706","volume":"85","author":"S Rosenthal","year":"1995","unstructured":"Rosenthal, S. & Chen, R. The reporting sensitivities of two passive surveillance systems for vaccine adverse events. Am. J. Public Health. 85, 1706\u20131709 (1995).","journal-title":"Am. J. Public Health."},{"key":"21986_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3310\/hta8250","volume":"8","author":"P Whiting","year":"2004","unstructured":"Whiting, P. et al. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technol. Assess. 8, 1\u2013234 (2004).","journal-title":"Health Technol. Assess."},{"key":"21986_CR33","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2020.602365","volume":"11","author":"JH Bae","year":"2021","unstructured":"Bae, J. H. et al. Machine learning for detection of safety signals from spontaneous reporting system data: Example of nivolumab and docetaxel. Front. Pharmacol. 11, 602365 (2021).","journal-title":"Front. Pharmacol."},{"key":"21986_CR34","unstructured":"Korea Institute of Drug Safety & Risk Management . Introduction of KAERS (Korea Adverse Event Reporting System) https:\/\/www.drugsafe.or.kr\/iwt\/ds\/en\/report\/WhatIsKAERS.do (2022)."},{"key":"21986_CR35","unstructured":"World Heatlth Organization. Anatomical Therapeutic Chemical (ATC) Classification https:\/\/www.who.int\/tools\/atc-ddd-toolkit\/atc-classification (2022)."},{"key":"21986_CR36","doi-asserted-by":"crossref","unstructured":"World Heatlth Organization. WHO-adverse reaction terminology (WHO-ART) in Dictionary of Pharmaceutical Medicine 192\u2013193 (Springer, 2009).","DOI":"10.1007\/978-3-211-89836-9_1467"},{"key":"21986_CR37","doi-asserted-by":"publisher","first-page":"796","DOI":"10.7150\/ijms.6048","volume":"10","author":"T Sakaeda","year":"2013","unstructured":"Sakaeda, T., Tamon, A., Kadoyama, K. & Okuno, Y. Data mining of the public version of the FDA adverse event reporting system. Int. J. Med. Sci. 10, 796\u2013803 (2013).","journal-title":"Int. J. Med. Sci."},{"key":"21986_CR38","unstructured":"TreeScan. TreeScan User Guide https:\/\/www.treescan.org\/techdoc.html (2022)."},{"key":"21986_CR39","unstructured":"Ministry of Food and Drug Safety. Integrated System for Safety Information of Drug https:\/\/nedrug.mfds.go.kr\/index (2022)."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-21986-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-21986-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-21986-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T07:43:12Z","timestamp":1669362192000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-21986-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,24]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["21986"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-21986-8","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,24]]},"assertion":[{"value":"28 June 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 November 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"J-YS received grants from the Ministry of Food and Drug Safety, the Ministry of Health and Welfare, the National Research Foundation of Korea, and pharmaceutical companies, including Daiichi Sankyo, GSK, and Pfizer, outside the submitted work. S-YJ received grants from the Ministry of Food and Drug Safety, the Ministry of Health and Welfare, the National Research Foundation of Korea. All other authors have no conflicts of interest to report.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"20256"}}