{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T02:01:01Z","timestamp":1772935261127,"version":"3.50.1"},"reference-count":37,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2021,2,17]],"date-time":"2021-02-17T00:00:00Z","timestamp":1613520000000},"content-version":"vor","delay-in-days":413,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Citation sentiment plays an important role in citation analysis and scholarly communication research, but prior citation sentiment studies have used small data sets and relied largely on manual annotation. This paper uses a large data set of PubMed Central (PMC) full-text publications and analyzes citation sentiment in more than 32 million citances within PMC, revealing citation sentiment patterns at the journal and discipline levels. This paper finds a weak relationship between a journal\u2019s citation impact (as measured by CiteScore) and the average sentiment score of citances to its publications. When journals are aggregated into quartiles based on citation impact, we find that journals in higher quartiles are cited more favorably than those in the lower quartiles. Further, social science journals are found to be cited with higher sentiment, followed by engineering and natural science and biomedical journals, respectively. This result may be attributed to disciplinary discourse patterns in which social science researchers tend to use more subjective terms to describe others\u2019 work than do natural science or biomedical researchers.<\/jats:p>","DOI":"10.1162\/qss_a_00040","type":"journal-article","created":{"date-parts":[[2020,3,25]],"date-time":"2020-03-25T09:30:56Z","timestamp":1585128656000},"page":"664-674","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":9,"title":["The relationship between journal citation impact and citation sentiment: A study of 32 million citances in PubMed Central"],"prefix":"10.1162","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0365-9340","authenticated-orcid":false,"given":"Erjia","family":"Yan","sequence":"first","affiliation":[{"name":"College of Computing and Informatics, Drexel University, 3675 Market Street, Philadelphia, PA 19104"}]},{"given":"Zheng","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computing and Informatics, Drexel University, 3675 Market Street, Philadelphia, PA 19104"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7264-365X","authenticated-orcid":false,"given":"Kai","family":"Li","sequence":"additional","affiliation":[{"name":"College of Computing and Informatics, Drexel University, 3675 Market Street, Philadelphia, PA 19104"}]}],"member":"281","published-online":{"date-parts":[[2020,6,1]]},"reference":[{"key":"2025073014022723300_bib1","unstructured":"Agarwal,  A., Xie,  B., Vovsha,  I., Rambow,  O., & Passonneau,  R. (2011). Sentiment analysis of Twitter data. In Proceedings of the Workshop on Language in Social Media (LSM 2011) (pp. 30\u201338). USA: Association for Computational Linguistics."},{"key":"2025073014022723300_bib2","unstructured":"Athar,  A.\n           (2011). Sentiment analysis of citations using sentence structure-based features. Paper presented at the Proceedings of the ACL 2011 student session."},{"key":"2025073014022723300_bib3","unstructured":"Athar,  A., & Teufel,  S. (2012). Context-enhanced citation sentiment detection. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 597\u2013601). USA: Association for Computational Linguistics."},{"key":"2025073014022723300_bib4","doi-asserted-by":"crossref","unstructured":"Bazerman,  C.\n           (1981). What written knowledge does: Three examples of academic discourse. Philosophy of the Social Sciences, 11(3), 361\u2013387.","DOI":"10.1177\/004839318101100305"},{"key":"2025073014022723300_bib5","unstructured":"Becher,  T., & Trowler,  P. (2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines. London: McGraw-Hill Education."},{"key":"2025073014022723300_bib6","unstructured":"Cambria,  E., Poria,  S., Bajpai,  R., & Schuller,  B. (2016). SenticNet 4: A semantic resource for sentiment analysis based on conceptual primitives. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics (pp. 2666\u20132677). Osaka: The COLING 2016 Organizing Committee."},{"key":"2025073014022723300_bib7","doi-asserted-by":"crossref","unstructured":"Case,  D. O., & Higgins,  G. M. (2000). How can we investigate citation behavior? A study of reasons for citing literature in communication. Journal of the Association for Information Science and Technology, 51(7), 635\u2013645.","DOI":"10.1002\/(SICI)1097-4571(2000)51:7<635::AID-ASI6>3.0.CO;2-H"},{"key":"2025073014022723300_bib8","unstructured":"Chen,  Z., & Yan,  E. (2017). Domain-independent term extraction & term network for scientific publications. iConference 2017 Proceedings (pp. 171\u2013189). Illinois: IDEALS. http:\/\/hdl.handle.net\/2142\/96671"},{"key":"2025073014022723300_bib9","doi-asserted-by":"crossref","unstructured":"Chubin,  D. E., & Moitra,  S. D. (1975). Content analysis of references: Adjunct or alternative to citation counting?Social Studies of Science, 5(4), 423\u2013441.","DOI":"10.1177\/030631277500500403"},{"key":"2025073014022723300_bib10","doi-asserted-by":"crossref","unstructured":"Demarest,  B., & Sugimoto,  C. R. (2015). Argue, observe, assess: Measuring disciplinary identities and differences through socio-epistemic discourse. Journal of the Association for Information Science and Technology, 66(7), 1374\u20131387.","DOI":"10.1002\/asi.23271"},{"key":"2025073014022723300_bib11","doi-asserted-by":"crossref","unstructured":"Erikson,  M. G., & Erlandson,  P. (2014). A taxonomy of motives to cite. Social Studies of Science, 44(4), 625\u2013637.","DOI":"10.1177\/0306312714522871"},{"key":"2025073014022723300_bib12","unstructured":"Garfield,  E.\n           (1965). Can citation indexing be automated? In Statistical Association Methods for Mechanized Documentation. Symposium Proceedings, Washington 1964. Washington, DC: US Government Printing Office."},{"key":"2025073014022723300_bib13","unstructured":"Garfield,  E., & Merton,  R. K. (1979). Citation indexing: Its theory and application in science, technology, and humanities (Vol. 8). Wiley: New York."},{"key":"2025073014022723300_bib14","doi-asserted-by":"crossref","unstructured":"Hicks,  D., Wouters,  P., Waltman,  L., De Rijcke,  S., & Rafols,  I. (2015). The Leiden Manifesto for research metrics. Nature, 520(7548), 429\u2013431.","DOI":"10.1038\/520429a"},{"key":"2025073014022723300_bib15","doi-asserted-by":"crossref","unstructured":"Jha,  R., Jbara,  A.-A., Qazvinian,  V., & Radev,  D. R. (2017). NLP-driven citation analysis for scientometrics. Natural Language Engineering, 23(1), 93\u2013130.","DOI":"10.1017\/S1351324915000443"},{"key":"2025073014022723300_bib16","unstructured":"Jurgens,  D., Kumar,  S., Hoover,  R., McFarland,  D., & Jurafsky,  D. (2016). Citation classification for behavioral analysis of a scientific field. arXiv:1609.00435."},{"key":"2025073014022723300_bib17","unstructured":"Merton,  R. K.\n           (1973). The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press."},{"key":"2025073014022723300_bib18","doi-asserted-by":"crossref","unstructured":"Montgomery,  S. L.\n           (2017). The Chicago guide to communicating science. Chicago: University of Chicago Press.","DOI":"10.7208\/chicago\/9780226144641.001.0001"},{"key":"2025073014022723300_bib19","doi-asserted-by":"crossref","unstructured":"Moravcsik,  M. J., & Murugesan,  P. (1975). Some results on the function and quality of citations. Social Studies of Science, 5(1), 86\u201392.","DOI":"10.1177\/030631277500500106"},{"key":"2025073014022723300_bib20","unstructured":"Nakov,  P. I., Schwartz,  A. S., & Hearst,  M. (2004). Citances: Citation sentences for semantic analysis of bioscience text. In Proceedings of the SIGIR \u201904 workshop on Search and Discovery in Bioinformatics. https:\/\/biotext.berkeley.edu\/papers\/citances-nlpbio04.pdf"},{"key":"2025073014022723300_bib21","doi-asserted-by":"crossref","unstructured":"Newman,  M. E.\n           (2004). Coauthorship networks and patterns of scientific collaboration. Proceedings of the National Academy of Sciences, 101(suppl 1), 5200\u20135205.","DOI":"10.1073\/pnas.0307545100"},{"key":"2025073014022723300_bib22","doi-asserted-by":"crossref","unstructured":"Pan,  X., Yan,  E., Wang,  Q., & Hua,  W. (2015). Assessing the impact of software on science: A bootstrapped learning of software entities in full-text papers. Journal of Informetrics, 9(4), 860\u2013871.","DOI":"10.1016\/j.joi.2015.07.012"},{"key":"2025073014022723300_bib23","doi-asserted-by":"crossref","unstructured":"Pang,  B., & Lee,  L. (2008). Opinion mining and sentiment analysis. Foundations and Trends\u00ae in Information Retrieval, 2(1\u20132), 1\u2013135.","DOI":"10.1561\/1500000011"},{"key":"2025073014022723300_bib24","doi-asserted-by":"crossref","unstructured":"Parry,  S.\n           (1998). Disciplinary discourse in doctoral theses. Higher Education, 36(3), 273\u2013299.","DOI":"10.1023\/A:1003216613001"},{"key":"2025073014022723300_bib25","doi-asserted-by":"crossref","unstructured":"PLOS Medicine Editors. (2006). The impact factor game. PLOS Med, 3(6), e291. https:\/\/doi.org\/10.1371\/journal.pmed.0030291","DOI":"10.1371\/journal.pmed.0030291"},{"key":"2025073014022723300_bib26","doi-asserted-by":"crossref","unstructured":"Small,  H.\n           (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265\u2013269.","DOI":"10.1002\/asi.4630240406"},{"key":"2025073014022723300_bib27","doi-asserted-by":"crossref","unstructured":"Small,  H.\n           (2004). On the shoulders of Robert Merton: Towards a normative theory of citation. Scientometrics, 60(1), 71\u201379.","DOI":"10.1023\/B:SCIE.0000027310.68393.bc"},{"key":"2025073014022723300_bib28","doi-asserted-by":"crossref","unstructured":"Small,  H. G.\n           (1978). Cited documents as concept symbols. Social Studies of Science, 8(3), 327\u2013340.","DOI":"10.1177\/030631277800800305"},{"key":"2025073014022723300_bib29","unstructured":"Teufel,  S.\n           (2009). Citations and sentiment. Paper presented at the Workshop on Text Mining for Scholarly Communications and Repositories, University of Manchester, UK."},{"key":"2025073014022723300_bib30","doi-asserted-by":"crossref","unstructured":"Teufel,  S., Siddharthan,  A., & Tidhar,  D. (2006). Automatic classification of citation function. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (pp. 103\u2013110). Sydney: Association for Computational Linguistics.","DOI":"10.3115\/1610075.1610091"},{"key":"2025073014022723300_bib31","unstructured":"Xu,  J., Zhang,  Y., Wu,  Y., Wang,  J., Dong,  X., & Xu,  H. (2015). Citation sentiment analysis in clinical trial papers. In AMIA Annual Symposium Proceedings (p. 1334). Bethesda, MD: American Medical Informatics Association."},{"key":"2025073014022723300_bib32","unstructured":"Yan,  E.\n           (2019). Journal sentiment data for 3728 Scopus journals. Retrieved from: https:\/\/doi.org\/10.6084\/m9.figshare.10247069.v1"},{"key":"2025073014022723300_bib33","doi-asserted-by":"crossref","unstructured":"Yan,  E., Chen,  Z., & Li,  K. (2020). Authors\u2019 status and the perceived quality of their work: Measuring citation sentiment change in Nobel articles. Journal of the Association for Information Science and Technology, 71(3), 314\u2013324.","DOI":"10.1002\/asi.24237"},{"key":"2025073014022723300_bib34","doi-asserted-by":"crossref","unstructured":"Yan,  E., & Ding,  Y. (2010). Weighted citation: An indicator of an article\u2019s prestige. Journal of the American Society for Information Science and Technology, 61(8), 1635\u20131643.","DOI":"10.1002\/asi.21349"},{"key":"2025073014022723300_bib35","unstructured":"Yan,  E., & Pan,  X. (2015). A bootstrapping method to assess software impact in full-text papers. In Proceedings of the 15th International Conference of the International Society for Scientometrics and Informetrics. Leuven: ISSI."},{"key":"2025073014022723300_bib36","doi-asserted-by":"crossref","unstructured":"Yan,  E., Williams,  J., & Chen,  Z. (2017). Understanding disciplinary vocabularies using a full-text enabled domain-independent term extraction approach. PLOS ONE, 12(11), e0187762.","DOI":"10.1371\/journal.pone.0187762"},{"key":"2025073014022723300_bib37","doi-asserted-by":"crossref","unstructured":"Zhang,  G., Ding,  Y., & Milojevi\u0107,  S. (2013). Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content. Journal of the American Society for Information Science and Technology, 64, 1490\u20131503.","DOI":"10.1002\/asi.22850"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/1\/2\/664\/1885806\/qss_a_00040.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/1\/2\/664\/1885806\/qss_a_00040.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T18:02:44Z","timestamp":1753898564000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/1\/2\/664\/96140\/The-relationship-between-journal-citation-impact"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020]]},"references-count":37,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,6,1]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00040","relation":{},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020]]},"published":{"date-parts":[[2020]]}}}