{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T21:55:30Z","timestamp":1777067730973,"version":"3.51.4"},"reference-count":23,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T00:00:00Z","timestamp":1761868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Res. Metr. Anal."],"abstract":"<jats:p>Traditional bibliometric approaches to research impact assessment have predominantly relied on citation counts, overlooking the qualitative dimensions of how research is received and discussed. Altmetrics have expanded this perspective by capturing mentions across diverse platforms, yet most analyses remain limited to quantitative measures, failing to account for sentiment. This study aimed to introduce a novel artificial intelligence-driven sentiment analysis framework designed to evaluate the tone and intent behind research mentions on social media, with a primary focus on X (formerly Twitter). Our approach leverages a bespoke sentiment classification system, spanning seven levels from strong negative to strong positive, to capture the nuanced ways in which research is endorsed, critiqued, or debated. Using a machine learning model trained on 5,732 manually curated labels (ML2024) as a baseline (F1 score = 0.419), we developed and refined a Large Language Model (LLM)-based classification system through three iterative rounds of expert evaluation. The final AI-driven model demonstrated improved alignment with human assessments, achieving an F1 score of 0.577, significantly enhancing precision and recall over traditional methods. These findings underscore the potential of advanced AI methodologies in altmetric analysis, offering a richer, more context-aware understanding of research reception. This study laid the foundation for integrating sentiment analysis into Altmetric platforms, providing researchers, institutions, and policymakers with deeper insights into the societal discourse surrounding scientific outputs.<\/jats:p>","DOI":"10.3389\/frma.2025.1612216","type":"journal-article","created":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T06:25:48Z","timestamp":1761891948000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Sentiment analysis of research attention: the Altmetric proof of concept"],"prefix":"10.3389","volume":"10","author":[{"given":"Carlos","family":"Areia","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Taylor","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miguel","family":"Garcia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Hernandez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,31]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e70525","DOI":"10.2196\/70525","article-title":"Sentiment analysis using a large language model\u2013based approach to detect opioids mixed with other substances via social media: method development and validation","volume":"5","author":"Ahmad","year":"2025","journal-title":"JMIR Infodemiol."},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1142\/S0218348X24400565","article-title":"Fractal-inspired sentiment analysis: evaluation of large language models and deep learning methods","author":"Alsagri","year":"2024","journal-title":"World Sci. Connect"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1188131","DOI":"10.3389\/frma.2023.1188131","article-title":"Evaluative altmetrics: is there evidence for its application to research evaluation?","volume":"8","author":"Arroyo-Machado","year":"2023","journal-title":"Front. Res. Metrics Anal."},{"key":"B4","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1007\/s11191-021-00233-y","article-title":"What do COVID-19 tweets reveal about public engagement with nature of science?","volume":"31","author":"Bichara","year":"2022","journal-title":"Sci. Educ."},{"key":"B5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.joi.2014.09.005","article-title":"Do altmetrics point to the broader impact of research? An overview of benef","volume":"8","author":"Bornmann","year":"2014","journal-title":"J. Informetr."},{"key":"B6","doi-asserted-by":"publisher","first-page":"23309","DOI":"10.1002\/asi.23309","article-title":"Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective","volume":"66","author":"Costas","year":"2014","journal-title":"J. Assoc. Inform. Sci. Technol."},{"key":"B7","unstructured":"Comprehensive study on sentiment analysis: from rule based to modern LLM based system\n          \n          \n            \n              Gupta\n              S.\n            \n            \n              Ranjan\n              R.\n            \n            \n              Singh\n              S. N.\n            \n          \n          arXiv\n          \n          2024"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1268","DOI":"10.1001\/jamapediatrics.2022.3581","article-title":"Detection of messenger RNA COVID-19 vaccines in human breast milk","volume":"176","author":"Hanna","year":"2022","journal-title":"JAMA Pediatr."},{"key":"B9","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.48550\/arxiv.2008.13023","article-title":"Exploiting tweet sentiments in altmetrics large-scale data","volume":"49","author":"Hassan","year":"2020","journal-title":"arXiv"},{"key":"B10","doi-asserted-by":"publisher","first-page":"e3003010","DOI":"10.1371\/journal.pbio.3003010","article-title":"A call for broadening the altmetrics tent to democratize science outreach","volume":"23","author":"Jari\u0107","year":"2025","journal-title":"PLoS Biol."},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/BigData62323.2024.10825350","article-title":"SentimentGPT: exploiting GPT for advanced sentiment analysis and its departure from current machine learning","author":"Kheiri","year":"2023","journal-title":"arXiv"},{"key":"B12","doi-asserted-by":"publisher","first-page":"e63631","DOI":"10.2196\/63631","article-title":"Large language models' accuracy in emulating human experts' evaluation of public sentiments about heated tobacco products on social media: evaluation study","volume":"27","author":"Kim","year":"2025","journal-title":"J. M\u00e9d. Internet Res."},{"key":"B13","doi-asserted-by":"publisher","first-page":"102723","DOI":"10.1016\/j.pmedr.2024.102723","article-title":"Accuracy of a large language model in distinguishing anti- and pro-vaccination messages on social media: the case of human papillomavirus vaccination","volume":"42","author":"Kim","year":"2024","journal-title":"Prev. Med. Rep."},{"key":"B14","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1002\/bult.2013.1720390410","article-title":"Five challenges in altmetrics: a toolmaker's perspective","volume":"39","author":"Liu","year":"2013","journal-title":"Bull. Am. Soc. Inform. Sci. Technol."},{"key":"B15","doi-asserted-by":"publisher","first-page":"9603","DOI":"10.1038\/s41598-024-60210-7","article-title":"A multimodal approach to cross-lingual sentiment analysis with ensemble of transformer and LLM","volume":"14","author":"Miah","year":"2024","journal-title":"Sci. Rep."},{"key":"B16","doi-asserted-by":"publisher","first-page":"2123","DOI":"10.1007\/s11192-018-2838-z","article-title":"Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data","volume":"116","author":"Ortega","year":"2018","journal-title":"Scientometrics"},{"key":"B17","doi-asserted-by":"publisher","first-page":"125450","DOI":"10.1016\/j.energy.2022.125450","article-title":"The role of new nuclear power in the UK's net-zero emissions energy system","volume":"262","author":"Price","year":"2023","journal-title":"Energy"},{"key":"B18","unstructured":"Priem\n              J.\n            \n          \n          As Scholars Undertake a Great Migration to Online Publishing, Altmetrics Stands to Provide an Academic Measurement of Twitter and Other Online Activity \u2014 Impact of Social Sciences\n          \n          2011"},{"key":"B19","doi-asserted-by":"publisher","first-page":"97","DOI":"10.2478\/jdis-2022-0003","article-title":"Public reaction to scientific research via twitter sentiment prediction","volume":"7","author":"Shahzad","year":"2021","journal-title":"J. Data Inf. Sci."},{"key":"B20","doi-asserted-by":"publisher","first-page":"2037","DOI":"10.1002\/asi.23833","article-title":"Scholarly use of social media and altmetrics: a review of the literature","volume":"68","author":"Sugimoto","year":"2017","journal-title":"J. Assoc. Inform. Sci. Technol."},{"key":"B21","doi-asserted-by":"publisher","first-page":"106026","DOI":"10.1016\/j.worlddev.2022.106026","article-title":"Capitalism and extreme poverty: a global analysis of real wages, human height, and mortality since the long 16th century","volume":"161","author":"Sullivan","year":"2023","journal-title":"World Dev."},{"key":"B22","doi-asserted-by":"publisher","first-page":"e0064841","DOI":"10.1371\/journal.pone.0064841","article-title":"Do altmetrics work? Twitter and ten other socialweb services","volume":"8","author":"Thelwall","year":"2013","journal-title":"PLoS ONE"},{"key":"B23","doi-asserted-by":"publisher","first-page":"587","DOI":"10.3145\/epi.2015.sep.08","article-title":"Web indicators for research evaluation. Part 1: citations and links to academic articles from the Web","volume":"24","author":"Thelwall","year":"2015","journal-title":"El Prof. Inform."}],"container-title":["Frontiers in Research Metrics and Analytics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2025.1612216\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T06:25:51Z","timestamp":1761891951000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2025.1612216\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,31]]},"references-count":23,"alternative-id":["10.3389\/frma.2025.1612216"],"URL":"https:\/\/doi.org\/10.3389\/frma.2025.1612216","relation":{},"ISSN":["2504-0537"],"issn-type":[{"value":"2504-0537","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,31]]},"article-number":"1612216"}}