{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T16:47:32Z","timestamp":1780418852164,"version":"3.54.1"},"reference-count":30,"publisher":"Proceedings of the National Academy of Sciences","issue":"48","license":[{"start":{"date-parts":[[2017,11,14]],"date-time":"2017-11-14T00:00:00Z","timestamp":1510617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["61672311"],"award-info":[{"award-number":["61672311"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["61532011"],"award-info":[{"award-number":["61532011"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.pnas.org"],"crossmark-restriction":true},"short-container-title":["Proc. Natl. Acad. Sci. U.S.A."],"published-print":{"date-parts":[[2017,11,28]]},"abstract":"<jats:title>Significance<\/jats:title>\n          <jats:p>Scientific peer review has been a cornerstone of the scientific method since the 1600s. Debate continues regarding the merits of single-blind review, in which anonymous reviewers know the authors of a paper and their affiliations, compared with double-blind review, in which this information is hidden. We present an experimental study of this question. In computer science, research often appears first or exclusively in peer-reviewed conferences rather than journals. Our study considers full-length submissions to the highly selective 2017 Web Search and Data Mining conference (15.6% acceptance rate). Each submission is simultaneously scored by two single-blind and two double-blind reviewers. Our analysis shows that single-blind reviewing confers a significant advantage to papers with famous authors and authors from high-prestige institutions.<\/jats:p>","DOI":"10.1073\/pnas.1707323114","type":"journal-article","created":{"date-parts":[[2017,11,14]],"date-time":"2017-11-14T16:50:37Z","timestamp":1510678237000},"page":"12708-12713","update-policy":"https:\/\/doi.org\/10.1073\/pnas.cm10313","source":"Crossref","is-referenced-by-count":422,"title":["Reviewer bias in single- versus double-blind peer review"],"prefix":"10.1073","volume":"114","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1611-9255","authenticated-orcid":false,"given":"Andrew","family":"Tomkins","sequence":"first","affiliation":[{"name":"Google, Inc., Mountain View, CA 94043;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William D.","family":"Heavlin","sequence":"additional","affiliation":[{"name":"Google, Inc., Mountain View, CA 94043;"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"341","published-online":{"date-parts":[[2017,11,14]]},"reference":[{"key":"e_1_3_3_1_2","unstructured":"A Tomkins M Zhang WD Heavlin Single versus double blind reviewing at WSDM 2017. arXiv:1702.00502. (2017)."},{"key":"e_1_3_3_2_2","volume-title":"How Professors Think: Inside the Curious World of Academic Judgment","author":"Lamont M","year":"2010","unstructured":"M Lamont How Professors Think: Inside the Curious World of Academic Judgment (Harvard Univ Press, Cambridge, MA, 2010)."},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/1168092.1168094"},{"key":"e_1_3_3_4_2","first-page":"75","volume-title":"Blinding as a Solution to Bias: Strengthening Biomedical Science, Forensic Science, and Law","author":"Largent EA","year":"2016","unstructured":"EA Largent, RT Snodgrass, Blind peer review by academic journals. Blinding as a Solution to Bias: Strengthening Biomedical Science, Forensic Science, and Law, eds C Robertson, A Kesselheim (Academic, Cambridge, MA), pp. 75\u201395 (2016)."},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/1206049.1206050"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/1416216.1416218"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1517480.1517492"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2015.00169"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tree.2007.07.008"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tree.2008.03.003"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/1147376.1147381"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1168092.1168093"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/980972.981001"},{"key":"e_1_3_3_14_2","unstructured":"S Jaschik Rejecting double blind. Available at https:\/\/www.insidehighered.com\/news\/2011\/05\/31\/american_economic_association_abandons_double_blind_journal_reviewing. Accessed January 29 2017. (2011)."},{"key":"e_1_3_3_15_2","first-page":"1041","article-title":"The effects of double-blind versus single-blind reviewing: Experimental evidence from the American economic review","volume":"81","author":"Blank RM","year":"1991","unstructured":"RM Blank, The effects of double-blind versus single-blind reviewing: Experimental evidence from the American economic review. Am Econ Rev 81, 1041\u20131067 (1991).","journal-title":"Am Econ Rev"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1093\/jole\/lzw009"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2016.11014"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1177\/1075547012472684"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.159.3810.56"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23349"},{"key":"e_1_3_3_21_2","volume-title":"Academic Careers for Experimental Computer Scientists and Engineers","author":"CORPORATE Committee on Academic Careers for Experimental Scientists and CORPORATE Commission on Physical Sciences, Mathematics & Applications","year":"1994","unstructured":"; CORPORATE Committee on Academic Careers for Experimental Scientists and CORPORATE Commission on Physical Sciences, Mathematics & Applications Academic Careers for Experimental Computer Scientists and Engineers (Natl Acad Press, Washington, DC, 1994)."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1511201113"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2979672"},{"key":"e_1_3_3_24_2","unstructured":"A Tomkins M Zhang 2017 call for papers. Available at www.wsdm-conference.org\/2017\/calls\/papers\/. Accessed January 29 2017. (2017)."},{"key":"e_1_3_3_25_2","unstructured":"U Luxburg I Guyon 2016 call for papers. Available at https:\/\/nips.cc\/Conferences\/2016\/CallForPapers. Accessed February 13 2017. (2016)."},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00011183"},{"key":"e_1_3_3_27_2","unstructured":"DP Peters SJ Ceci The Peters & Ceci study of journal publications. Available at https:\/\/thewinnower.com\/discussions\/7-_the-_peters-_ceci-_study-_of-_journal-_publications. Accessed January 29 2017. (2014)."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1093\/brain\/123.9.1964"},{"key":"e_1_3_3_29_2","unstructured":"N Lawrence Get your NIPS reviews in! Available at inverseprobability.com\/2015\/07\/23\/get-your-nips-review-in. Accessed January 29 2017. (2015)."},{"key":"e_1_3_3_30_2","unstructured":"N Lawrence NIPS experiment analysis. Available at inverseprobability.com\/2015\/03\/30\/nips-experiment-analysis. Accessed January 29 2017. (2015)."}],"container-title":["Proceedings of the National Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.pnas.org\/syndication\/doi\/10.1073\/pnas.1707323114","content-type":"unspecified","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pnas.org\/doi\/pdf\/10.1073\/pnas.1707323114","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,13]],"date-time":"2022-04-13T06:14:26Z","timestamp":1649830466000},"score":1,"resource":{"primary":{"URL":"https:\/\/pnas.org\/doi\/full\/10.1073\/pnas.1707323114"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,14]]},"references-count":30,"journal-issue":{"issue":"48","published-print":{"date-parts":[[2017,11,28]]}},"alternative-id":["10.1073\/pnas.1707323114"],"URL":"https:\/\/doi.org\/10.1073\/pnas.1707323114","relation":{},"ISSN":["0027-8424","1091-6490"],"issn-type":[{"value":"0027-8424","type":"print"},{"value":"1091-6490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,14]]},"assertion":[{"value":"2017-11-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}