{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T09:08:25Z","timestamp":1774429705844,"version":"3.50.1"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2020,12,22]],"date-time":"2020-12-22T00:00:00Z","timestamp":1608595200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"National Center for Research Resources and the National Center for Advancing Translational Sciences of the National Institutes of Health","award":["UL1TR001414"],"award-info":[{"award-number":["UL1TR001414"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,6,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Sentiment analysis is a popular tool for analyzing health-related social media content. However, existing studies exhibit numerous methodological issues and inconsistencies with respect to research design and results reporting, which could lead to biased data, imprecise or incorrect conclusions, or incomparable results across studies. This article reports a systematic analysis of the literature with respect to such issues. The objective was to develop a standardized protocol for improving the research validity and comparability of results in future relevant studies.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We developed the Protocol of Analysis of senTiment in Health (PATH) based on a systematic review that analyzed common research design choices and how such choices were made, or reported, among eligible studies published 2010-2019.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Of 409 articles screened, 89 met the inclusion criteria. A total of 16 distinctive research design choices were identified, 9 of which have significant methodological or reporting inconsistencies among the articles reviewed, ranging from how relevance of study data was determined to how the sentiment analysis tool selected was validated. Based on this result, we developed the PATH protocol that encompasses all these distinctive design choices and highlights the ones for which careful consideration and detailed reporting are particularly warranted.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>A substantial degree of methodological and reporting inconsistencies exist in the extant literature that applied sentiment analysis to analyzing health-related social media data. The PATH protocol developed through this research may contribute to mitigating such issues in future relevant studies.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa298","type":"journal-article","created":{"date-parts":[[2020,12,5]],"date-time":"2020-12-05T12:21:03Z","timestamp":1607170863000},"page":"1125-1134","source":"Crossref","is-referenced-by-count":17,"title":["Developing a standardized protocol for computational sentiment analysis research using health-related social media data"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0181-018X","authenticated-orcid":false,"given":"Lu","family":"He","sequence":"first","affiliation":[{"name":"Department of Informatics, Donald Bren School of Information and Computer Science, University of California, Irvine, Irvine, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tingjue","family":"Yin","sequence":"additional","affiliation":[{"name":"Department of Informatics, Donald Bren School of Information and Computer Science, University of California, Irvine, Irvine, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaoxian","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Informatics, Donald Bren School of Information and Computer Science, University of California, Irvine, Irvine, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yunan","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Informatics, Donald Bren School of Information and Computer Science, University of California, Irvine, Irvine, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David A","family":"Hanauer","sequence":"additional","affiliation":[{"name":"Department of Learning Health Sciences, School of Medicine, University of Michigan, Ann Arbor, Michigan, USA"},{"name":"Department of Pediatrics, School of Medicine, University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Informatics, Donald Bren School of Information and Computer Science, University of California, Irvine, Irvine, California, USA"},{"name":"Department of Emergency Medicine, School of Medicine, University of California, Irvine, Irvine, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,12,22]]},"reference":[{"key":"2021061318591780800_ocaa298-B1","first-page":"64: 1","author":"Pruksachatkun","year":"2019"},{"issue":"9","key":"2021061318591780800_ocaa298-B2","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.1080\/10410236.2017.1339370","article-title":"Sentiment analysis of an online breast cancer support group: communicating about tamoxifen","volume":"33","author":"Cabling","year":"2018","journal-title":"Health Commun"},{"issue":"5","key":"2021061318591780800_ocaa298-B3","doi-asserted-by":"crossref","first-page":"e167","DOI":"10.2196\/jmir.6946","article-title":"Public response to Obamacare on Twitter","volume":"19","author":"Davis","year":"2017","journal-title":"J Med Internet Res"},{"key":"2021061318591780800_ocaa298-B4","author":"Thelwall","year":"2020"},{"issue":"1","key":"2021061318591780800_ocaa298-B5","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s13326-017-0120-6","article-title":"Optimization on machine learning based approaches for sentiment analysis on HPV vaccines related tweets","volume":"8","author":"Du","year":"2017","journal-title":"J Biomed Semantics"},{"issue":"S2","key":"2021061318591780800_ocaa298-B6","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1186\/s12911-017-0469-6","article-title":"Leveraging machine learning-based approaches to assess human papillomavirus vaccination sentiment trends with Twitter data","volume":"17","author":"Du","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"key":"2021061318591780800_ocaa298-B7","author":"Shepherd"},{"key":"2021061318591780800_ocaa298-B8"},{"issue":"1\u20132","key":"2021061318591780800_ocaa298-B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/1500000011","article-title":"Opinion mining and sentiment analysis","volume":"2","author":"Pang","year":"2008","journal-title":"FNT Inf Retriev"},{"key":"2021061318591780800_ocaa298-B10","first-page":"1","author":"Liu","year":"2012"},{"issue":"4","key":"2021061318591780800_ocaa298-B11","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1097\/HMR.0000000000000154","article-title":"Predicting HCAHPS scores from hospitals\u2019 social media pages: a sentiment analysis","volume":"43","author":"Huppertz","year":"2018","journal-title":"Health Care Manage Rev"},{"issue":"1","key":"2021061318591780800_ocaa298-B12","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1177\/0261927X09351676","article-title":"The psychological meaning of words: LIWC and computerized text analysis methods","volume":"29","author":"Tausczik","year":"2010","journal-title":"J Lang Soc Psychol"},{"key":"2021061318591780800_ocaa298-B13","author":"Baccianella","year":"2010"},{"key":"2021061318591780800_ocaa298-B14","first-page":"55","author":"Manning","year":"2014"},{"key":"2021061318591780800_ocaa298-B15","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.ijinfomgt.2017.12.002","article-title":"Social media analytics \u2013 challenges in topic discovery, data collection, and data preparation","volume":"39","author":"Stieglitz","year":"2018","journal-title":"Int J Inf Manag"},{"issue":"1","key":"2021061318591780800_ocaa298-B16","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.artmed.2015.03.006","article-title":"Sentiment analysis in medical settings: new opportunities and challenges","volume":"64","author":"Denecke","year":"2015","journal-title":"Artif Intell Med"},{"key":"2021061318591780800_ocaa298-B17","first-page":"1208","article-title":"How do general-purpose sentiment analyzers perform when applied to health-related online social media data?","volume":"264","author":"He","year":"2019","journal-title":"Stud Health Technol Inform"},{"key":"2021061318591780800_ocaa298-B18","first-page":"440","author":"Blitzer","year":"2007"},{"issue":"6","key":"2021061318591780800_ocaa298-B19","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1080\/10810730.2018.1493057","article-title":"Okay, we get it. You vape\u201d: an analysis of geocoded content, context, and sentiment regarding e-cigarettes on Twitter","volume":"23","author":"Martinez","year":"2018","journal-title":"J Health Commun"},{"key":"2021061318591780800_ocaa298-B20","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.ijmedinf.2018.10.002","article-title":"Utilizing Twitter data for analysis of chemotherapy","volume":"120","author":"Zhang","year":"2018","journal-title":"Int J Med Inform"},{"issue":"3","key":"2021061318591780800_ocaa298-B21","doi-asserted-by":"crossref","first-page":"357","DOI":"10.2105\/AJPH.2019.305461","article-title":"Cannabis surveillance with twitter data: emerging topics and social bots","volume":"110","author":"Allem","year":"2020","journal-title":"Am J Public Health"},{"issue":"6","key":"2021061318591780800_ocaa298-B22","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1093\/jamia\/ocz009","article-title":"A systematic literature review of machine learning in online personal health data","volume":"26","author":"Yin","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2021061318591780800_ocaa298-B23","doi-asserted-by":"crossref","first-page":"e85","DOI":"10.2196\/jmir.1933","article-title":"A new dimension of health care: systematic review of the uses, benefits, and limitations of social media for health communication","volume":"15","author":"Moorhead","year":"2013","journal-title":"J Med Internet Res"},{"issue":"2","key":"2021061318591780800_ocaa298-B24","doi-asserted-by":"crossref","first-page":"e43","DOI":"10.2196\/publichealth.5789","article-title":"Sentiment analysis of health care tweets: review of the methods used","volume":"4","author":"Gohil","year":"2018","journal-title":"JMIR Public Health Surveill"},{"issue":"CSCW","key":"2021061318591780800_ocaa298-B25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3359249","article-title":"Who is the \u201chuman\u201d in human-centered machine learning: the case of predicting mental health from social media","volume":"3","author":"Chancellor","year":"2019","journal-title":"Proc ACM Hum-Comput Interact"},{"issue":"1","key":"2021061318591780800_ocaa298-B26","doi-asserted-by":"crossref","first-page":"e16023","DOI":"10.2196\/16023","article-title":"Sentiment analysis in health and well-being: systematic review","volume":"8","author":"Zunic","year":"2020","journal-title":"JMIR Med Inform"},{"issue":"4","key":"2021061318591780800_ocaa298-B27","doi-asserted-by":"crossref","first-page":"264","DOI":"10.7326\/0003-4819-151-4-200908180-00135","article-title":"Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement","volume":"151","author":"Moher","year":"2009","journal-title":"Ann Intern Med"},{"key":"2021061318591780800_ocaa298-B28","volume-title":"Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory","author":"Corbin","year":"2014"},{"issue":"6","key":"2021061318591780800_ocaa298-B29","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1016\/j.jbi.2013.08.011","article-title":"Text classification for assisting moderators in online health communities","volume":"46","author":"Huh","year":"2013","journal-title":"J Biomed Inform"},{"issue":"12","key":"2021061318591780800_ocaa298-B30","doi-asserted-by":"crossref","first-page":"e414","DOI":"10.2196\/jmir.9266","article-title":"Using social media data to understand the impact of promotional information on laypeople\u2019s discussions: a case study of lynch syndrome","volume":"19","author":"Bian","year":"2017","journal-title":"J Med Internet Res"},{"key":"2021061318591780800_ocaa298-B31","first-page":"197","author":"Yuan","year":"2018"},{"issue":"7","key":"2021061318591780800_ocaa298-B32","doi-asserted-by":"crossref","first-page":"e0200800","DOI":"10.1371\/journal.pone.0200800","article-title":"Social interactions in online eating disorder communities: a network perspective","volume":"13","author":"Wang","year":"2018","journal-title":"PLoS One"},{"issue":"7","key":"2021061318591780800_ocaa298-B33","doi-asserted-by":"crossref","first-page":"e0181233","DOI":"10.1371\/journal.pone.0181233","article-title":"A novel surveillance approach for disaster mental health","volume":"12","author":"Gruebner","year":"2017","journal-title":"PLoS One"},{"issue":"e2","key":"2021061318591780800_ocaa298-B34","doi-asserted-by":"crossref","first-page":"e212","DOI":"10.1136\/amiajnl-2013-002282","article-title":"Finding influential users of online health communities: a new metric based on sentiment influence","volume":"21","author":"Zhao","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2021061318591780800_ocaa298-B35","doi-asserted-by":"crossref","first-page":"e11036","DOI":"10.2196\/11036","article-title":"Identifying key topics bearing negative sentiment on Twitter: insights concerning the 2015-2016 zika epidemic","volume":"5","author":"Mamidi","year":"2019","journal-title":"JMIR Public Health Surveill"},{"key":"2021061318591780800_ocaa298-B36","first-page":"1089","author":"Roccetti","year":"2015"},{"key":"2021061318591780800_ocaa298-B37","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.jbi.2014.03.006","article-title":"Pharmaceutical drugs chatter on Online Social Networks","volume":"49","author":"Wiley","year":"2014","journal-title":"J Biomed Inform"},{"issue":"8\u20139","key":"2021061318591780800_ocaa298-B38","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1093\/jamia\/ocz056","article-title":"Mapping gender transition sentiment patterns via social media data: toward decreasing transgender mental health disparities","volume":"26","author":"Haimson","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"8","key":"2021061318591780800_ocaa298-B39","doi-asserted-by":"crossref","first-page":"e219","DOI":"10.2196\/jmir.6185","article-title":"The importance of debiasing social media data to better understand e-cigarette-related attitudes and behaviors","volume":"18","author":"Allem","year":"2016","journal-title":"J Med Internet Res"},{"issue":"10","key":"2021061318591780800_ocaa298-B40","doi-asserted-by":"crossref","first-page":"1378","DOI":"10.2105\/AJPH.2018.304567","article-title":"Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate","volume":"108","author":"Broniatowski","year":"2018","journal-title":"Am J Public Health"},{"issue":"2","key":"2021061318591780800_ocaa298-B41","doi-asserted-by":"crossref","first-page":"e41","DOI":"10.2196\/jmir.4738","article-title":"Garbage in, garbage out: data collection, quality assessment and reporting standards for social media data use in health research, infodemiology and digital disease detection","volume":"18","author":"Kim","year":"2016","journal-title":"J Med Internet Res"},{"issue":"3","key":"2021061318591780800_ocaa298-B42","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1177\/0022042619833911","article-title":"Choosing your platform for social media drug research and improving your keyword filter list","volume":"49","author":"Adams","year":"2019","journal-title":"J Drug Issues"},{"key":"2021061318591780800_ocaa298-B43","first-page":"703","author":"Hogenboom","year":"2013"},{"key":"2021061318591780800_ocaa298-B44","first-page":"1211","author":"Lu","year":"2015"},{"issue":"2","key":"2021061318591780800_ocaa298-B45","doi-asserted-by":"crossref","first-page":"e162","DOI":"10.2196\/publichealth.6327","article-title":"When \u2018bad\u2019 is \u2018good\u2019\u201d: identifying personal communication and sentiment in drug-related tweets","volume":"2","author":"Daniulaityte","year":"2016","journal-title":"JMIR Public Health Surveill"},{"key":"2021061318591780800_ocaa298-B46","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/j.jbi.2015.11.004","article-title":"Crowdsourcing Twitter annotations to identify first-hand experiences of prescription drug use","volume":"58","author":"Alvaro","year":"2015","journal-title":"J Biomed Inform"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1125\/38615362\/ocaa298.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1125\/38615362\/ocaa298.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,13]],"date-time":"2021-06-13T18:59:58Z","timestamp":1623610798000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/6\/1125\/6045013"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,22]]},"references-count":46,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,12,22]]},"published-print":{"date-parts":[[2021,6,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa298","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,6,1]]},"published":{"date-parts":[[2020,12,22]]}}}