{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T11:35:43Z","timestamp":1761824143999,"version":"3.37.3"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2019,12,30]],"date-time":"2019-12-30T00:00:00Z","timestamp":1577664000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM009886-10","5T15LM007079-27"],"award-info":[{"award-number":["R01LM009886-10","5T15LM007079-27"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Scientific commentaries are expected to play an important role in evidence appraisal, but it is unknown whether this expectation has been fulfilled. This study aims to better understand the role of scientific commentary in evidence appraisal. We queried PubMed for all clinical research articles with accompanying comments and extracted corresponding metadata. Five percent of clinical research studies (N\u2009=\u2009130\u00a0629) received postpublication comments (N\u2009=\u2009171\u00a0556), resulting in 178\u00a0882 comment\u2013article pairings, with 90% published in the same journal. We obtained 5197 full-text comments for topic modeling and exploratory sentiment analysis. Topics were generally disease specific with only a few topics relevant to the appraisal of studies, which were highly prevalent in letters. Of a random sample of 518 full-text comments, 67% had a supportive tone. Based on our results, published commentary, with the exception of letters, most often highlight or endorse previous publications rather than serve as a prominent mechanism for critical appraisal.<\/jats:p>","DOI":"10.1093\/jamia\/ocz209","type":"journal-article","created":{"date-parts":[[2019,11,23]],"date-time":"2019-11-23T20:09:26Z","timestamp":1574539766000},"page":"449-456","source":"Crossref","is-referenced-by-count":11,"title":["Understanding the nature and scope of clinical research commentaries in PubMed"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7329-3535","authenticated-orcid":false,"given":"James R","family":"Rogers","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hollis","family":"Mills","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1153-1590","authenticated-orcid":false,"given":"Lisa V","family":"Grossman","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrew","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Department of Medicine, New York University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9624-0214","authenticated-orcid":false,"given":"Chunhua","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2019,12,30]]},"reference":[{"issue":"9912","key":"2020110613110346600_ocz209-B1","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/S0140-6736(13)62227-8","article-title":"Increasing value and reducing waste in research design, conduct, and analysis","volume":"383","author":"Ioannidis","year":"2014","journal-title":"Lancet"},{"issue":"8","key":"2020110613110346600_ocz209-B2","doi-asserted-by":"crossref","first-page":"e124.","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLOS Med"},{"issue":"9453","key":"2020110613110346600_ocz209-B3","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/S0140-6736(04)17670-8","article-title":"External validity of randomised controlled trials: to whom do the results of this trial apply?","volume":"365","author":"Rothwell","year":"2005","journal-title":"Lancet"},{"issue":"11","key":"2020110613110346600_ocz209-B4","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1001\/jama.282.11.1061","article-title":"Empirical evidence of design-related bias in studies of diagnostic tests","volume":"282","author":"Lijmer","year":"1999","journal-title":"JAMA"},{"issue":"1","key":"2020110613110346600_ocz209-B5","doi-asserted-by":"crossref","first-page":"113.","DOI":"10.1186\/s12875-017-0680-7","article-title":"Clinical drug trials in general practice: how well are external validity issues reported?","volume":"18","author":"Br\u00e6nd","year":"2017","journal-title":"BMC Fam Pract"},{"issue":"6","key":"2020110613110346600_ocz209-B6","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1093\/jamia\/ocx050","article-title":"Evidence appraisal: a scoping review, conceptual framework, and research agenda","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2020110613110346600_ocz209-B7","doi-asserted-by":"crossref","first-page":"e01256\u201316","DOI":"10.1128\/mBio.01256-16","article-title":"A framework for improving the quality of research in the biological sciences","volume":"7","author":"Casadevall","year":"2016","journal-title":"mBio"},{"issue":"2","key":"2020110613110346600_ocz209-B8","doi-asserted-by":"crossref","first-page":"161","DOI":"10.4155\/cli.14.116","article-title":"Data fraud in clinical trials","volume":"5","author":"George","year":"2015","journal-title":"Clin Investig"},{"issue":"6","key":"2020110613110346600_ocz209-B9","doi-asserted-by":"crossref","first-page":"429","DOI":"10.7326\/0003-4819-157-6-201209180-00537","article-title":"Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials","volume":"157","author":"Savovi\u0107","year":"2012","journal-title":"Ann Intern Med"},{"issue":"4","key":"2020110613110346600_ocz209-B10","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1016\/j.jclinepi.2010.07.017","article-title":"GRADE guidelines: 4. Rating the quality of evidence\u2014study limitations (risk of bias)","volume":"64","author":"Guyatt","year":"2011","journal-title":"J Clin Epidemiol"},{"issue":"7597","key":"2020110613110346600_ocz209-B11","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1136\/bmj.39136.682083.AE","article-title":"Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials","volume":"334","author":"Ferreira-Gonz\u00e1lez","year":"2007","journal-title":"BMJ"},{"issue":"3","key":"2020110613110346600_ocz209-B12","doi-asserted-by":"crossref","first-page":"722","DOI":"10.1016\/j.jsurg.2017.07.026","article-title":"What makes journal club effective?\u2014a survey of orthopaedic residents and faculty","volume":"75","author":"Campbell","year":"2018","journal-title":"J Surg Educ"},{"issue":"1","key":"2020110613110346600_ocz209-B13","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.jsurg.2011.07.004","article-title":"Teaching evidence based medicine to surgery residents-is journal club the best format? A systematic review of the literature","volume":"69","author":"Ahmadi","year":"2012","journal-title":"J Surg Educ"},{"issue":"8","key":"2020110613110346600_ocz209-B14","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1016\/j.nedt.2010.12.020","article-title":"Exploring the impact of journal clubs: a systematic review","volume":"31","author":"Honey","year":"2011","journal-title":"Nurse Educ Today"},{"issue":"1","key":"2020110613110346600_ocz209-B15","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1056\/NEJMp048043","article-title":"Journal clubs\u2014science as conversation","volume":"351","author":"Wright","year":"2004","journal-title":"N Engl J Med"},{"issue":"1","key":"2020110613110346600_ocz209-B16","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1191\/0269216302pm517oa","article-title":"Daily journal club: an education tool in palliative care","volume":"16","author":"Mazuryk","year":"2002","journal-title":"Palliat Med"},{"issue":"5","key":"2020110613110346600_ocz209-B17","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.tig.2015.03.006","article-title":"Reviewing post-publication peer review","volume":"31","author":"Knoepfler","year":"2015","journal-title":"Trends Genet"},{"issue":"2","key":"2020110613110346600_ocz209-B18","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1016\/j.neuron.2014.03.032","article-title":"The Vacuum Shouts Back: post publication peer review on social media","volume":"82","author":"Faulkes","year":"2014","journal-title":"Neuron"},{"key":"2020110613110346600_ocz209-B19","doi-asserted-by":"crossref","DOI":"10.3389\/fncom.2012.00018","article-title":"Learning from open source software projects to improve scientific review","author":"Ghosh","year":"2012","journal-title":"Front Comput Neurosci"},{"issue":"9","key":"2020110613110346600_ocz209-B20","doi-asserted-by":"crossref","first-page":"2089","DOI":"10.1007\/s00405-014-3289-7","article-title":"What is the role of \u2018the letter to the editor\u2019?","volume":"272","author":"Tierney","year":"2015","journal-title":"Eur Arch Otorhinolaryngol"},{"issue":"12","key":"2020110613110346600_ocz209-B21","doi-asserted-by":"crossref","first-page":"904.","DOI":"10.1503\/cmaj.109-4817","article-title":"When postpublication peer review stings","volume":"186","author":"Collier","year":"2014","journal-title":"CMAJ"},{"issue":"21","key":"2020110613110346600_ocz209-B22","doi-asserted-by":"crossref","first-page":"2843","DOI":"10.1001\/jama.287.21.2843","article-title":"Postpublication criticism and the shaping of clinical knowledge","volume":"287","author":"Horton","year":"2002","journal-title":"JAMA"},{"issue":"16","key":"2020110613110346600_ocz209-B23","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1001\/jama.281.16.1543","article-title":"Letters: a forum for scientific discourse","volume":"281","author":"Winker","year":"1999","journal-title":"JAMA"},{"year":"2019","key":"2020110613110346600_ocz209-B24"},{"year":"2018","key":"2020110613110346600_ocz209-B25"},{"issue":"1","key":"2020110613110346600_ocz209-B26","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1186\/s13643-016-0259-8","article-title":"Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews","volume":"5","author":"J\u00f8rgensen","year":"2016","journal-title":"Syst Rev"},{"issue":"1","key":"2020110613110346600_ocz209-B27","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1186\/1756-0500-6-414","article-title":"What do letters to the editor publish about randomized controlled trials? A cross-sectional study","volume":"6","author":"Kastner","year":"2013","journal-title":"BMC Res Notes"},{"key":"2020110613110346600_ocz209-B28","doi-asserted-by":"crossref","first-page":"c3926.","DOI":"10.1136\/bmj.c3926","article-title":"Adequacy of authors\u2019 replies to criticism raised in electronic letters to the editor: cohort study","volume":"341","author":"G\u00f8tzsche","year":"2010","journal-title":"BMJ"},{"issue":"11","key":"2020110613110346600_ocz209-B29","doi-asserted-by":"crossref","first-page":"1422","DOI":"10.1093\/bioinformatics\/btp163","article-title":"Biopython: freely available Python tools for computational molecular biology and bioinformatics","volume":"25","author":"Cock","year":"2009","journal-title":"Bioinformatics"},{"year":"2017","author":"Sayers","key":"2020110613110346600_ocz209-B30"},{"year":"2018","author":"Chamberlain","key":"2020110613110346600_ocz209-B31"},{"year":"2018","author":"Richardson","key":"2020110613110346600_ocz209-B32"},{"year":"2014","author":"Shinyama","key":"2020110613110346600_ocz209-B33"},{"issue":"4","key":"2020110613110346600_ocz209-B34","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1145\/2133806.2133826","article-title":"Probabilistic topic models","volume":"55","author":"Blei","year":"2012","journal-title":"Commun ACM"},{"key":"2020110613110346600_ocz209-B35","first-page":"993","article-title":"Latent Dirichlet allocation","volume":"3","author":"Blei","year":"2003","journal-title":"J Mach Learn Res"},{"key":"2020110613110346600_ocz209-B36","first-page":"1","article-title":"topicmodels: an R package for fitting topic models","volume":"40","author":"Hornik","year":"2011","journal-title":"J Stat Softw"},{"issue":"6","key":"2020110613110346600_ocz209-B37","first-page":"1400","article-title":"Biomedical text mining for research rigor and integrity: tasks, challenges, directions","volume":"19","author":"Kilicoglu","year":"2018","journal-title":"Brief Bioinform"},{"key":"2020110613110346600_ocz209-B38","first-page":"1334","article-title":"Citation sentiment analysis in clinical trial papers","volume":"2015","author":"Xu","year":"2015","journal-title":"AMIA Ann Symp Proc"},{"issue":"1","key":"2020110613110346600_ocz209-B39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/meet.14505001084","article-title":"Automated citation sentiment analysis: what can we learn from biomedical researchers","volume":"50","author":"Yu","year":"2013","journal-title":"Proc Am Soc Inf Sci Technol"},{"issue":"12","key":"2020110613110346600_ocz209-B40","doi-asserted-by":"crossref","first-page":"e1001772.","DOI":"10.1371\/journal.pmed.1001772","article-title":"A stronger post-publication culture is needed for better science","volume":"11","author":"Bastian","year":"2014","journal-title":"PLoS Med"},{"year":"2019","author":"Moody","key":"2020110613110346600_ocz209-B41"},{"key":"2020110613110346600_ocz209-B42","first-page":"147","volume-title":"Advances in Neural Information Processing Systems 18","author":"Lafferty","year":"2006"},{"issue":"1\/2","key":"2020110613110346600_ocz209-B43","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1023\/A:1007617005950","article-title":"Unsupervised learning by probabilistic latent semantic analysis","volume":"42","author":"Hofmann","year":"2001","journal-title":"Mach Learn"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/3\/449\/34152706\/ocz209.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/3\/449\/34152706\/ocz209.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:36:35Z","timestamp":1604691395000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/3\/449\/5691202"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,30]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,12,30]]},"published-print":{"date-parts":[[2020,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz209","relation":{},"ISSN":["1527-974X"],"issn-type":[{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2020,3]]},"published":{"date-parts":[[2019,12,30]]}}}