{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T20:39:36Z","timestamp":1760647176857},"reference-count":19,"publisher":"Cambridge University Press (CUP)","issue":"2","license":[{"start":{"date-parts":[[2020,3,4]],"date-time":"2020-03-04T00:00:00Z","timestamp":1583280000000},"content-version":"unspecified","delay-in-days":3,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2020,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The ACL-2019 Business meeting ended with a discussion of reviewing. Conferences are experiencing a success catastrophe. They are becoming bigger and bigger, which is not only a sign of success but also a challenge (for reviewing and more). Various proposals for reducing submissions were discussed at the Business meeting. IMHO, the problem is not so much too many submissions, but rather, random reviewing. We cannot afford to do reviewing as badly as we do (because that leads to even more submissions). Negative feedback loops are effective. The reviewing process will improve over time if reviewers teach authors how to write better submissions, and authors teach reviewers how to write more constructive reviews. If you have received a not-ok (unhelpful\/offensive) review, please help program committees improve by sharing your not-ok reviews on social media.<\/jats:p>","DOI":"10.1017\/s1351324920000030","type":"journal-article","created":{"date-parts":[[2020,3,4]],"date-time":"2020-03-04T05:43:20Z","timestamp":1583300600000},"page":"245-257","source":"Crossref","is-referenced-by-count":5,"title":["Emerging trends: Reviewing the reviewers (again)"],"prefix":"10.1017","volume":"26","author":[{"given":"Kenneth Ward","family":"Church","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2020,3,4]]},"reference":[{"key":"S1351324920000030_ref19","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2007.33.4.607"},{"key":"S1351324920000030_ref12","unstructured":"Page, L. , Brin, S. , Motwani, R. and Winograd, T. (1999). The PageRank citation ranking: Bringing order to the web. Technical report, Stanford University, Stanford, CA."},{"key":"S1351324920000030_ref9","doi-asserted-by":"publisher","DOI":"10.1038\/546009a"},{"key":"S1351324920000030_ref8","unstructured":"Klovanis, M. , Porcher, R. , Ravaud, P. and Trinquart, L. (2016) The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PloS One, e0166387."},{"key":"S1351324920000030_ref7","doi-asserted-by":"publisher","DOI":"10.1093\/biosci\/bix034"},{"key":"S1351324920000030_ref5","doi-asserted-by":"publisher","DOI":"10.1186\/s41073-017-0027-x"},{"key":"S1351324920000030_ref4","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324917000286"},{"key":"S1351324920000030_ref2","doi-asserted-by":"publisher","DOI":"10.1162\/089120105775299131"},{"key":"S1351324920000030_ref10","first-page":"313","article-title":"Building a large annotated corpus of English: The Penn Treebank","volume":"19","author":"Marcus","year":"(1993)","journal-title":"Computational Linguistics"},{"key":"S1351324920000030_ref16","unstructured":"Tennant, J. , Dugan, J. et al. (2017). A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research, 6, Faculty of 1000 Ltd."},{"key":"S1351324920000030_ref20","unstructured":"Yarowsky, D. and Radu, F. (1999). Taking the load off the conference chairs-towards a digital paper-routing assistant. In EMNLP."},{"key":"S1351324920000030_ref6","doi-asserted-by":"publisher","DOI":"10.2307\/2943969"},{"key":"S1351324920000030_ref17","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1707323114"},{"key":"S1351324920000030_ref15","unstructured":"Stelmakh, I. , Shah, N. and Singh, A. (2019). On testing for biases in peer review. In Advances in Neural Information Processing Systems (NeurIPS), vol. 32, pp. 5287\u20135297."},{"key":"S1351324920000030_ref14","doi-asserted-by":"publisher","DOI":"10.1186\/bcr2742"},{"key":"S1351324920000030_ref11","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-017-0927-0"},{"key":"S1351324920000030_ref13","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"S1351324920000030_ref1","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-015-0395-3"},{"key":"S1351324920000030_ref18","volume-title":"Infant Mortality and the European Demographic Transition","author":"Van de Walle","year":"(1986)"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324920000030","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,3,4]],"date-time":"2020-03-04T05:43:43Z","timestamp":1583300623000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324920000030\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3]]},"references-count":19,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,3]]}},"alternative-id":["S1351324920000030"],"URL":"https:\/\/doi.org\/10.1017\/s1351324920000030","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3]]}}}