{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T19:11:42Z","timestamp":1784229102620,"version":"3.55.0"},"reference-count":65,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[2020,2,12]],"date-time":"2020-02-12T00:00:00Z","timestamp":1581465600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["OIR"],"published-print":{"date-parts":[[2020,2,12]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title><jats:p>The purpose of this study is to investigate the relationship of the post-publication peer review (PPPR) polarity of a paper to that paper's citation count.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title><jats:p>Papers with PPPRs from Publons.com as the experimental groups were manually matched 1:2 with the related papers without PPPR as the control group, by the same journal, the same issue (volume), the same access status (gold open access or not) and the same document type. None of the papers in the experimental group or control group received any comments or recommendations from ResearchGate, PubPeer or F1000. The polarity of the PPPRs was coded by using content analysis. A negative binomial regression analysis was conducted to examine the data by controlling the characteristics of papers.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Findings<\/jats:title><jats:p>The four experimental groups and their corresponding control groups were generated as follows: papers with neutral PPPRs, papers with both negative and positive PPPRs, papers with negative PPPRs and papers with positive PPPRs as well as four corresponding control groups (papers without PPPRs). The results are as follows: while holding the other variables (such as page count, number of authors, etc.) constant in the model, papers that received neutral PPPRs, those that received negative PPPRs and those that received both negative and positive PPPRs had no significant differences in citation count when compared to their corresponding control pairs (papers without PPPRs). Papers that received positive PPPRs had significantly greater citation count than their corresponding control pairs (papers without PPPRs) while holding the other variables (such as page count, number of authors, etc.) constant in the model.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title><jats:p>Based on a broader range of PPPR sentiments, by controlling many of the confounding factors (including the characteristics of the papers and the effects of the other PPPR platforms), this study analyzed the relationship of various polarities of PPPRs to citation count.<\/jats:p><\/jats:sec>","DOI":"10.1108\/oir-01-2019-0027","type":"journal-article","created":{"date-parts":[[2020,2,24]],"date-time":"2020-02-24T08:50:01Z","timestamp":1582534201000},"page":"583-602","source":"Crossref","is-referenced-by-count":12,"title":["The relationship of polarity of post-publication peer review to citation count"],"prefix":"10.1108","volume":"44","author":[{"given":"Qianjin","family":"Zong","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lili","family":"Fan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yafen","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingshi","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","reference":[{"issue":"9","key":"key2020061010571725400_ref001","first-page":"1118","article-title":"Impact of article page count and number of authors on citations in disability related fields: a systematic review article","volume":"45","year":"2016","journal-title":"Iranian Journal of Public Health"},{"issue":"1","key":"key2020061010571725400_ref002","first-page":"17","article-title":"Citations, citation indicators, and research quality: an overview of basic concepts and theories","volume":"9","year":"2019","journal-title":"Sage Open"},{"issue":"7","key":"key2020061010571725400_ref003","first-page":"1","article-title":"Social media release increases dissemination of original articles in the clinical pain sciences","volume":"8","year":"2013","journal-title":"PloS One"},{"issue":"2","key":"key2020061010571725400_ref004","article-title":"Bibliometric analysis of the top-cited gastroenterology and hepatology articles","volume":"6","year":"2016","journal-title":"BMJ Open"},{"issue":"1","key":"key2020061010571725400_ref005","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1007\/s11192-017-2242-0","article-title":"Post retraction citations in context: a case study","volume":"113","year":"2017","journal-title":"Scientometrics"},{"issue":"1","key":"key2020061010571725400_ref006","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1186\/s12874-018-0606-7","article-title":"Reliability in evaluator-based tests: using simulation-constructed models to determine contextually relevant agreement thresholds","volume":"18","year":"2018","journal-title":"BMC Medical Research Methodology"},{"issue":"1","key":"key2020061010571725400_ref007","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1080\/13697137.2016.1250739","article-title":"What is wrong with the medical literature?","volume":"20","year":"2017","journal-title":"Climacteric"},{"issue":"6","key":"key2020061010571725400_ref008","doi-asserted-by":"crossref","first-page":"1984","DOI":"10.3758\/s13428-016-0838-6","article-title":"Reliability in content analysis: the case of semantic feature norms classification","volume":"49","year":"2017","journal-title":"Behavior Research Methods"},{"issue":"3","key":"key2020061010571725400_ref009","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1016\/j.joi.2015.04.001","article-title":"Which people use which scientific papers? An evaluation of data from F1000 and Mendeley","volume":"9","year":"2015","journal-title":"Journal of Informetrics"},{"issue":"5","key":"key2020061010571725400_ref010","first-page":"12","article-title":"Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data","volume":"13","year":"2018","journal-title":"PloS One"},{"issue":"2","key":"key2020061010571725400_ref011","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1016\/j.joi.2012.12.003","article-title":"The validation of (advanced) bibliometric indicators through peer assessments: a comparative study using data from InCites and F1000","volume":"7","year":"2013","journal-title":"Journal of Informetrics"},{"issue":"3","key":"key2020061010571725400_ref012","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1016\/j.joi.2015.03.001","article-title":"Does quality and content matter for citedness? A comparison with para-textual factors and over time","volume":"9","year":"2015","journal-title":"Journal of Informetrics"},{"issue":"6","key":"key2020061010571725400_ref013","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1002\/asi.23100","article-title":"The wisdom of citing scientists","volume":"65","year":"2014","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"key2020061010571725400_ref014","article-title":"PubMed commons: a system for commenting on articles in PubMed","year":"2013"},{"issue":"45","key":"key2020061010571725400_ref015","doi-asserted-by":"crossref","first-page":"13823","DOI":"10.1073\/pnas.1502280112","article-title":"The incidence and role of negative citations in science","volume":"112","year":"2015","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"issue":"1","key":"key2020061010571725400_ref016","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1139\/facets-2016-0032","article-title":"Open access articles receive more citations in hybrid marine ecology journals","volume":"2","year":"2017","journal-title":"FACETS"},{"key":"key2020061010571725400_ref017","first-page":"1","article-title":"The need for post-publication peer review in plant science publishing","volume":"4","year":"2013","journal-title":"Frontiers in Plant Science"},{"issue":"1","key":"key2020061010571725400_ref018","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/s11948-016-9863-1","article-title":"Reflection on the Fazlul Sarkar versus PubPeer (\u2018John Doe\u2019) case","volume":"24","year":"2018","journal-title":"Science and Engineering Ethics"},{"issue":"2","key":"key2020061010571725400_ref019","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1108\/OIR-04-2018-0139","article-title":"Challenges to open peer review","volume":"43","year":"2019","journal-title":"Online Information Review"},{"issue":"1","key":"key2020061010571725400_ref020","first-page":"22","article-title":"Problems with traditional science publishing and finding a wider niche for post-publication peer review","volume":"22","year":"2015","journal-title":"Accountability in Research-Policies and Quality Assurance"},{"issue":"6","key":"key2020061010571725400_ref021","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1002\/asi.23934","article-title":"On the differences between citations and Altmetrics: an investigation of factors driving Altmetrics versus citations for Finnish articles","volume":"69","year":"2018","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"key2020061010571725400_ref022","article-title":"The largest database of peer-reviewed literature - Scopus - Elsevier solutions","year":"2018"},{"issue":"10","key":"key2020061010571725400_ref023","first-page":"1","article-title":"The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations","volume":"11","year":"2013","journal-title":"PLoS Biology"},{"issue":"15","key":"key2020061010571725400_ref024","doi-asserted-by":"crossref","first-page":"4024","DOI":"10.1158\/0008-5472.CAN-14-0925","article-title":"The increasing urgency for standards in basic biologic research","volume":"74","year":"2014","journal-title":"Cancer Research"},{"issue":"1","key":"key2020061010571725400_ref025","first-page":"10","article-title":"ReCal OIR: ordinal, interval, and ratio intercoder reliability as a web service","volume":"8","year":"2013","journal-title":"International Journal of Internet Science"},{"issue":"4","key":"key2020061010571725400_ref026","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1007\/BF02019306","article-title":"Is citation analysis a legitimate evaluation tool?","volume":"1","year":"1979","journal-title":"Scientometrics"},{"issue":"3","key":"key2020061010571725400_ref027","doi-asserted-by":"crossref","first-page":"1573","DOI":"10.1007\/s11192-016-2216-7","article-title":"Title characteristics and citations in economics","volume":"110","year":"2017","journal-title":"Scientometrics"},{"issue":"2","key":"key2020061010571725400_ref028","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1007\/s11192-013-1176-4","article-title":"The power of book reviews: a simple and transparent enhancement approach for book citation indexes","volume":"98","year":"2014","journal-title":"Scientometrics"},{"issue":"2","key":"key2020061010571725400_ref029","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1111\/j.1365-2753.2011.01646.x","article-title":"Understanding belief using citation networks","volume":"17","year":"2011","journal-title":"Journal of Evaluation in Clinical Practice"},{"key":"key2020061010571725400_ref030","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.jpsychires.2018.07.010","article-title":"Bibliometric analysis of manuscript characteristics that influence citations: a comparison of six major psychiatry journals","volume":"108","year":"2019","journal-title":"Journal of Psychiatric Research"},{"issue":"4","key":"key2020061010571725400_ref031","doi-asserted-by":"crossref","first-page":"670","DOI":"10.1016\/j.mayocp.2018.09.004","article-title":"Peer review bias: a critical review","volume":"94","year":"2019","journal-title":"Mayo Clinic Proceedings"},{"issue":"1","key":"key2020061010571725400_ref032","doi-asserted-by":"crossref","first-page":"9","DOI":"10.6087\/kcse.2014.1.9","article-title":"The changing face of peer review","volume":"1","year":"2014","journal-title":"Science Editing"},{"issue":"1","key":"key2020061010571725400_ref033","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.joi.2014.11.003","article-title":"The value of experience in research","volume":"9","year":"2015","journal-title":"Journal of Informetrics"},{"key":"key2020061010571725400_ref034","first-page":"1","article-title":"Post-publication peer review: opening up scientific conversation","volume":"6","year":"2012","journal-title":"Frontiers in Computational Neuroscience"},{"key":"key2020061010571725400_ref035","first-page":"1","article-title":"The culture of scientific research","volume":"4","year":"2015","journal-title":"F1000Research"},{"issue":"5","key":"key2020061010571725400_ref036","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.tig.2015.03.006","article-title":"Reviewing post-publication peer review","volume":"31","year":"2015","journal-title":"Trends in Genetics"},{"key":"key2020061010571725400_ref037","first-page":"1","article-title":"Toward a new model of scientific publishing: discussion and a proposal","volume":"5","year":"2011","journal-title":"Frontiers in Computational Neuroscience"},{"issue":"3","key":"key2020061010571725400_ref038","first-page":"411","article-title":"Reliability in content analysis","volume":"30","year":"2004","journal-title":"Human Communication Research"},{"issue":"1","key":"key2020061010571725400_ref039","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1111\/vru.12168","article-title":"Acceptance rate and reasons for rejection of manuscripts submitted to veterinary radiology & ultrasound during 2012","volume":"56","year":"2015","journal-title":"Veterinary Radiology & Ultrasound"},{"key":"key2020061010571725400_ref040","first-page":"26","article-title":"Adoption of PubMed commons as a forum for post-publication peer-review","volume":"31","year":"2015","journal-title":"Current Medical Research and Opinion"},{"key":"key2020061010571725400_ref041","unstructured":"Li, X. and Thelwall, M. (2012), \u201cF1000, Mendeley and traditional bibliometric indicators\u201d, in Proceedings of the 17th International Conference on Science and Technology Indicators, Vol. 2, pp. 451-551."},{"issue":"11","key":"key2020061010571725400_ref042","doi-asserted-by":"crossref","first-page":"e259","DOI":"10.2196\/jmir.2707","article-title":"Correlation and interaction visualization of altmetric indicators extracted from scholarly social network activities: dimensions and structure","volume":"15","year":"2013","journal-title":"Journal of Medical Internet Research"},{"issue":"7148","key":"key2020061010571725400_ref043","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1038\/4471052d","article-title":"Why are people reluctant to join in open review?","volume":"447","year":"2007","journal-title":"Nature"},{"issue":"6","key":"key2020061010571725400_ref044","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1177\/0267659116667265","article-title":"Traditional peer review and post-publication peer review","volume":"31","year":"2016","journal-title":"Perfusion"},{"issue":"2","key":"key2020061010571725400_ref045","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1629\/uksg.245","article-title":"Post-publication peer review, in all its guises, is here to stay","volume":"28","year":"2015","journal-title":"Insights"},{"issue":"2","key":"key2020061010571725400_ref046","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1007\/s11192-013-0993-9","article-title":"Assessing non-standard article impact using F1000 labels","volume":"97","year":"2013","journal-title":"Scientometrics"},{"key":"key2020061010571725400_ref047","first-page":"1","article-title":"Do peer review models affect clinicians' trust in journals? A survey of junior doctors","volume":"2","year":"2017","journal-title":"Research Integrity and Peer Review"},{"issue":"5","key":"key2020061010571725400_ref048","article-title":"Citation analysis of scientific categories","volume":"3","year":"2017","journal-title":"Heliyon"},{"issue":"1","key":"key2020061010571725400_ref049","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1111\/oik.02956","article-title":"Will technology trample peer review in ecology? Ongoing issues and potential solutions","volume":"125","year":"2016","journal-title":"OIKOS"},{"issue":"6","key":"key2020061010571725400_ref050","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1002\/2327-6924.12389","article-title":"Post publication peer review: is it coming to your nursing journal?","volume":"28","year":"2016","journal-title":"Journal of the American Association of Nurse Practitioners"},{"issue":"1","key":"key2020061010571725400_ref051","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/s10746-014-9335-z","article-title":"From manuscript evaluation to article valuation: the changing technologies of journal peer review","volume":"38","year":"2015","journal-title":"Human Studies"},{"issue":"6","key":"key2020061010571725400_ref052","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.2214\/AJR.14.13944","article-title":"Metrics for original research articles in the AJR: from first submission to final publication","volume":"204","year":"2015","journal-title":"American Journal of Roentgenology"},{"issue":"1","key":"key2020061010571725400_ref053","first-page":"1","article-title":"Post-publication peer review in biomedical journals: overcoming obstacles and disincentives to knowledge sharing","volume":"6","year":"2018","journal-title":"Research Policy and Evaluation"},{"issue":"1","key":"key2020061010571725400_ref054","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.gie.2019.03.028","article-title":"Longitudinal relationship between social media activity and article citations in the journal Gastrointestinal Endoscopy","volume":"90","year":"2019","journal-title":"Gastrointestinal Endoscopy"},{"issue":"1","key":"key2020061010571725400_ref055","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1108\/JD-06-2017-0092","article-title":"\u2018Let the community decide\u2019? The vision and reality of soundness-only peer review in open-access mega-journals","volume":"74","year":"2018","journal-title":"Journal of Documentation"},{"issue":"6","key":"key2020061010571725400_ref056","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1038\/onc.2017.390","article-title":"The importance of being earnest in post-publication review: scientific fraud and the scourges of anonymity and excuses","volume":"37","year":"2018","journal-title":"Oncogene"},{"issue":"5","key":"key2020061010571725400_ref057","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1108\/OIR-06-2015-0182","article-title":"For what it's worth - the open peer review landscape","volume":"39","year":"2015","journal-title":"Online Information Review"},{"issue":"9","key":"key2020061010571725400_ref058","doi-asserted-by":"crossref","first-page":"2089","DOI":"10.1007\/s00405-014-3289-7","article-title":"What is the role of \u2018the letter to the editor\u2019?","volume":"272","year":"2015","journal-title":"European Archives of Oto-Rhino-Laryngology"},{"issue":"2","key":"key2020061010571725400_ref059","doi-asserted-by":"crossref","first-page":"147","DOI":"10.3109\/09540261.2014.998991","article-title":"Social media, medicine and the modern journal club","volume":"27","year":"2015","journal-title":"International Review of Psychiatry"},{"key":"key2020061010571725400_ref060","first-page":"1","article-title":"Towards an open science publishing platform","volume":"5","year":"2016","journal-title":"F1000Research"},{"issue":"4","key":"key2020061010571725400_ref061","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1016\/j.joi.2016.10.004","article-title":"The impact of author-selected keywords on citation counts","volume":"10","year":"2016","journal-title":"Journal of Informetrics"},{"issue":"5","key":"key2020061010571725400_ref062","first-page":"452","article-title":"Publishing your work in a journal: understanding the peer review process","volume":"7","year":"2012","journal-title":"International Journal of Sports Physical Therapy"},{"issue":"3","key":"key2020061010571725400_ref063","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1002\/asi.23040","article-title":"F1000 recommendations as a potential new data source for research evaluation: a comparison with citations","volume":"65","year":"2014","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"key2020061010571725400_ref064","first-page":"11","article-title":"Do \u2018Faculty of 1000\u2019(F1000) ratings of ecological publications serve as reasonable predictors of their future impact?","volume":"3","year":"2010","journal-title":"Ideas in Ecology and Evolution"},{"issue":"2","key":"key2020061010571725400_ref065","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1087\/20150209","article-title":"The promise of post-publication peer review: how do we get there from here?","volume":"28","year":"2015","journal-title":"Learned Publishing"}],"container-title":["Online Information Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/OIR-01-2019-0027\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/OIR-01-2019-0027\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:42:03Z","timestamp":1753396923000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/oir\/article\/44\/3\/583-602\/452514"}},"subtitle":["Evidence from Publons"],"short-title":[],"issued":{"date-parts":[[2020,2,12]]},"references-count":65,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,2,12]]}},"alternative-id":["10.1108\/OIR-01-2019-0027"],"URL":"https:\/\/doi.org\/10.1108\/oir-01-2019-0027","relation":{},"ISSN":["1468-4527"],"issn-type":[{"value":"1468-4527","type":"print"}],"subject":[],"published":{"date-parts":[[2020,2,12]]}}}