{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,5]],"date-time":"2026-09-05T03:54:18Z","timestamp":1788580458537,"version":"build-2803163510"},"reference-count":94,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Vaccines serve as a crucial public health tool, although vaccine hesitancy continues to pose a significant threat to full vaccine uptake and, consequently, community health. Understanding and tracking vaccine hesitancy is essential for effective public health interventions; however, traditional survey methods present various limitations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to create a real-time, natural language processing (NLP)\u2013based tool to assess vaccine sentiment and hesitancy across 3 prominent social media platforms.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We mined and curated discussions in English from Twitter (subsequently rebranded as X), Reddit, and YouTube social media platforms posted between January 1, 2011, and October 31, 2021, concerning human papillomavirus; measles, mumps, and rubella; and unspecified vaccines. We tested multiple NLP algorithms to classify vaccine sentiment into positive, neutral, or negative and to classify vaccine hesitancy using the World Health Organization\u2019s (WHO) 3Cs (confidence, complacency, and convenience) hesitancy model, conceptualizing an online dashboard to illustrate and contextualize trends.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We compiled over 86 million discussions. Our top-performing NLP models displayed accuracies ranging from 0.51 to 0.78 for sentiment classification and from 0.69 to 0.91 for hesitancy classification. Explorative analysis on our platform highlighted variations in online activity about vaccine sentiment and hesitancy, suggesting unique patterns for different vaccines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Our innovative system performs real-time analysis of sentiment and hesitancy on 3 vaccine topics across major social networks, providing crucial trend insights to assist campaigns aimed at enhancing vaccine uptake and public health.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/57164","type":"journal-article","created":{"date-parts":[[2024,4,11]],"date-time":"2024-04-11T01:42:36Z","timestamp":1712799756000},"page":"e57164","source":"Crossref","is-referenced-by-count":13,"title":["Natural Language Processing\u2013Powered Real-Time Monitoring Solution for Vaccine Sentiments and Hesitancy on Social Media: System Development and Validation"],"prefix":"10.2196","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5661-8940","authenticated-orcid":false,"given":"Liang-Chin","family":"Huang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9422-1215","authenticated-orcid":false,"given":"Amanda L","family":"Eiden","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8842-1841","authenticated-orcid":false,"given":"Long","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2696-6765","authenticated-orcid":false,"given":"Augustine","family":"Annan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-3323-4803","authenticated-orcid":false,"given":"Siwei","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-4574-1448","authenticated-orcid":false,"given":"Jingqi","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1030-6348","authenticated-orcid":false,"given":"Frank J","family":"Manion","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4193-4120","authenticated-orcid":false,"given":"Xiaoyan","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0322-4566","authenticated-orcid":false,"given":"Jingcheng","family":"Du","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5187-6120","authenticated-orcid":false,"given":"Lixia","family":"Yao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2024,6,21]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(22)00320-6"},{"key":"ref2","unstructured":"LindmeierCMeasles vaccination has saved an estimated 17.1 million lives since 2000World Health Organization20152024-05-08https:\/\/www.who.int\/news\/item\/12-11-2015-measles-vaccination-has-saved-an-estimated-17-1-million-lives-since-2000"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1016\/s0264-410x(02)00623-0"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2015.04.036"},{"key":"ref5","unstructured":"Ten threats to global health in 2019World Health Organization2024-05-08https:\/\/www.who.int\/news-room\/spotlight\/ten-threats-to-global-health-in-2019"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1586\/14760584.2015.964212"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2018.03.063"},{"issue":"3-4","key":"ref8","first-page":"351","volume":"55","author":"Black, FL","year":"1982","journal-title":"Yale J Biol Med"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.d5703"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2014-2715"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1001\/jama.296.14.1757"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2011-0400"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2013.07.013"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1016\/j.amepre.2009.10.035"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6874-10-27"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1016\/j.amepre.2015.10.002"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1371\/currents.outbreaks.6844c80ff9f5b273f34c91f71b7fc289"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1007\/s40124-022-00278-9"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(13)70108-7"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1007\/s10912-014-9278-4"},{"key":"ref21","unstructured":"WHO TThe guide to tailoring immunization programmesWHO Regional Office for Europe20132024-05-08https:\/\/iris.who.int\/handle\/10665\/351166"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2014.04.018"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1037\/a0016942"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1363\/4403912"},{"issue":"3","key":"ref25","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1542\/peds.93.3.369","volume":"93","author":"Clayton, EW","year":"1994","journal-title":"Pediatrics"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.7393"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1177\/0956797614557867"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0135072"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v10i1.14729"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-018-1247-8"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.1933"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2016.11.007"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.2196\/publichealth.5059"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajic.2012.10.016"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2009.11.060"},{"key":"ref36","first-page":"603","author":"Chowdhary, KR","year":"2020","journal-title":"Fundamentals of Artificial Intelligence"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1088\/1751-8113\/44\/8\/085201"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1002199"},{"key":"ref39","unstructured":"Vaccine Sentiments on Social Media2024-05-08https:\/\/vaccine.social\/"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118715"},{"key":"ref41","doi-asserted-by":"publisher","DOI":"10.1186\/s12889-022-12824-7"},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.1136\/bmjgh-2021-008334"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2023.107734"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.3390\/vaccines10101713"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-018-1236-4"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1007\/s13278-021-00776-6"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1177\/2051013614557476"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1016\/j.coviro.2019.01.002"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1080\/14760584.2023.2217257"},{"key":"ref50","author":"Kleinbaum, DG","year":"2002","journal-title":"Logistic Regression: A Self-Learning Text"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1038\/nbt1206-1565"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1080\/01431160412331269698"},{"key":"ref53","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.1213\/ANE.0000000000005247"},{"key":"ref56","doi-asserted-by":"publisher","DOI":"10.1155\/2018\/4302425"},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1016\/j.imu.2019.100180"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.32604\/cmc.2021.016727"},{"key":"ref59","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-020-03542-1"},{"key":"ref60","unstructured":"RamosJUsing TF-IDF to determine word relevance in document queriesProceedings of the first instructional conference on machine learning20032024-05-13https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=b3bf6373ff41a115197 cb5b30e57830c16130c2c"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2008.2002909"},{"issue":"3","key":"ref63","first-page":"9","volume":"1","author":"Kirasich, K","year":"2018","journal-title":"SMU Data Science Review"},{"key":"ref64","doi-asserted-by":"publisher","DOI":"10.3390\/a15080291"},{"key":"ref65","unstructured":"HensleySWorried parents balk at HPV vaccine for daughtersNPR20132024-05-08https:\/\/www.npr.org\/sections\/health-shots\/2013\/03\/18\/174617709\/worried-parents-balk-at-hpv-vaccine-for-daughters"},{"key":"ref66","unstructured":"CastilloMSide effect fears stop parents from getting HPV vaccine for daughtersCBS News20132024-05-08https:\/\/www.cbsnews.com\/news\/side-effect-fears-stop-parents-from-getting-hpv-vaccine-for-daughters\/"},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.1016\/j.ypmed.2013.05.013"},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.1080\/21645515.2018.1564436"},{"key":"ref69","unstructured":"HPV, the vaccine for HPV, and cancers caused by HPVCenters for Disease Control and Prevention20222024-05-08https:\/\/tinyurl.com\/2d45j3jz"},{"key":"ref70","doi-asserted-by":"publisher","DOI":"10.15585\/mmwr.mm6832a3"},{"key":"ref71","unstructured":"BranswellHThird dose of mumps vaccine could help stop outbreaks, researchers saySTAT20172024-05-08https:\/\/www.statnews.com\/2017\/09\/06\/mumps-vaccine-study\/"},{"key":"ref72","unstructured":"ScuttiSCDC recommends booster shot of MMR vaccine during mumps outbreaksCNN Health20172024-05-08https:\/\/www.cnn.com\/2017\/10\/25\/health\/cdc-mumps-outbreak-syracuse-university\/index.html"},{"key":"ref73","unstructured":"GoldschmidtDJournal questions validity of autism and vaccine studyCNN Health20142024-05-08https:\/\/www.cnn.com\/2014\/08\/27\/health\/irpt-cdc-autism-vaccine-study\/index.html"},{"key":"ref74","unstructured":"ParkAWhistleblower claims CDC covered up data showing vaccine-autism linkTIME20142024-05-08https:\/\/time.com\/3208886\/whistleblower-claims-cdc-covered-up-data-showing-vaccine-autism-link\/"},{"key":"ref75","unstructured":"Autism and vaccinesCenters for Disease Control and Prevention20212024-05-08https:\/\/www.cdc.gov\/vaccinesafety\/concerns\/autism.html"},{"key":"ref76","unstructured":"Epidemiological WWMMR and autismWorld Health Organization20032024-05-08https:\/\/www.who.int\/groups\/global-advi sory-committee-on-vaccine-safety\/topics\/mmr-vaccines-and-autism"},{"key":"ref77","doi-asserted-by":"publisher","DOI":"10.1016\/s0140-6736(10)60175-4"},{"key":"ref78","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2021.08.052"},{"key":"ref79","doi-asserted-by":"crossref","first-page":"4367","DOI":"10.18653\/v1\/2021.emnlp-main.358","author":"Park, S","year":"2021","journal-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"},{"key":"ref80","doi-asserted-by":"publisher","DOI":"10.1145\/3489141"},{"key":"ref81","doi-asserted-by":"publisher","DOI":"10.1109\/taslp.2019.2959251"},{"key":"ref82","doi-asserted-by":"publisher","DOI":"10.1109\/tkde.2015.2489653"},{"key":"ref83","doi-asserted-by":"crossref","first-page":"292","DOI":"10.18653\/v1\/W18-6243","author":"Xu, P","year":"2018","journal-title":"Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis"},{"key":"ref84","unstructured":"YeZLiFBaldwinTEncoding sentiment information into word vectors for sentiment analysis2018Proceedings of the 27th International Conference on Computational LinguisticsAugust, 2018Santa Fe, New Mexico, USAAssociation for Computational Linguistics"},{"key":"ref85","doi-asserted-by":"publisher","DOI":"10.1109\/taslp.2017.2788182"},{"key":"ref86","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.107663"},{"key":"ref87","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3006531"},{"key":"ref88","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2017.11.003"},{"key":"ref89","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2017.04.060"},{"key":"ref90","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.22025"},{"key":"ref91","doi-asserted-by":"publisher","DOI":"10.1016\/j.tele.2023.101983"},{"key":"ref92","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2022.104804"},{"key":"ref93","doi-asserted-by":"publisher","DOI":"10.4135\/9781412963947.n376"},{"key":"ref94","doi-asserted-by":"publisher","DOI":"10.15195\/v6.a20"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2024\/1\/e57164","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:47:09Z","timestamp":1782841629000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2024\/1\/e57164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,21]]},"references-count":94,"URL":"https:\/\/doi.org\/10.2196\/57164","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.57164","asserted-by":"object"}]},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,21]]}}}