{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,5]],"date-time":"2025-10-05T19:55:06Z","timestamp":1759694106245,"version":"3.41.0"},"reference-count":25,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2012,1,9]],"date-time":"2012-01-09T00:00:00Z","timestamp":1326067200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCOMM Comput. Commun. Rev."],"published-print":{"date-parts":[[2013,1,9]]},"abstract":"<jats:p>The computer science research paper review process is largely human and time-intensive. More worrisome, review processes are frequently questioned, and often non-transparent. This work advocates applying computer science methods and tools to the computer science review process. As an initial exploration, we data mine the submissions, bids, reviews, and decisions from a recent top-tier computer networking conference. We empirically test several common hypotheses, including the existence of readability, citation, call-for-paper adherence, and topical bias. From our findings, we hypothesize review process methods to improve fairness, efficiency, and transparency.<\/jats:p>","DOI":"10.1145\/2427036.2427040","type":"journal-article","created":{"date-parts":[[2013,1,11]],"date-time":"2013-01-11T15:42:48Z","timestamp":1357918968000},"page":"22-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Findings and implications from data mining the IMC review process"],"prefix":"10.1145","volume":"43","author":[{"given":"Robert","family":"Beverly","sequence":"first","affiliation":[{"name":"Naval Postgraduate School, Monterey, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Allman","sequence":"additional","affiliation":[{"name":"International Computer Science Institute, Berkeley, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,1,9]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"SIGCOMM Internet Measurement Conference","author":"ACM","year":"2010","unstructured":"ACM SIGCOMM Internet Measurement Conference , 2010 . http:\/\/www.sigcomm.org\/events\/imc-conference. ACM SIGCOMM Internet Measurement Conference, 2010. http:\/\/www.sigcomm.org\/events\/imc-conference."},{"key":"e_1_2_1_2_1","unstructured":"IEEE ComSoc Technical Committee on Computer Communications mailing list archives Aug. 2010. https:\/\/lists.cs.columbia.edu\/pipermail\/tccc\/2010-August\/thread.html. IEEE ComSoc Technical Committee on Computer Communications mailing list archives Aug. 2010. https:\/\/lists.cs.columbia.edu\/pipermail\/tccc\/2010-August\/thread.html."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1355734.1355741"},{"key":"e_1_2_1_4_1","volume-title":"WOWCS","author":"Allman M.","year":"2008","unstructured":"M. Allman . What ought a program committee to do ? In WOWCS , 2008 . M. Allman. What ought a program committee to do? In WOWCS, 2008."},{"key":"e_1_2_1_5_1","volume-title":"Networking Conferences Statistics","author":"Almeroth K.","year":"2012","unstructured":"K. Almeroth . Networking Conferences Statistics , 2012 . http:\/\/www.cs.ucsb.edu\/~almeroth\/conf\/stats\/. K. Almeroth. Networking Conferences Statistics, 2012. http:\/\/www.cs.ucsb.edu\/~almeroth\/conf\/stats\/."},{"key":"e_1_2_1_6_1","volume-title":"WOWCS","author":"Anderson T. E.","year":"2008","unstructured":"T. E. Anderson . Towards a model of computer systems research . In WOWCS , 2008 . T. E. Anderson. Towards a model of computer systems research. In WOWCS, 2008."},{"key":"e_1_2_1_7_1","volume-title":"Retrieval techniques. Annual Review of Information Science and Technology (ARIST), 22:109--145","author":"Belkin N.","year":"1987","unstructured":"N. Belkin and W. Croft . Retrieval techniques. Annual Review of Information Science and Technology (ARIST), 22:109--145 , 1987 . N. Belkin and W. Croft. Retrieval techniques. Annual Review of Information Science and Technology (ARIST), 22:109--145, 1987."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1506409.1506421"},{"key":"e_1_2_1_9_1","volume-title":"Openness of the SIGCOMM conference, 2008","author":"Crovella M.","year":"2008","unstructured":"M. Crovella . Openness of the SIGCOMM conference, 2008 . http:\/\/blog.sigcomm.org\/ 2008 \/09\/openness_of_the_sigcomm_confer.html. M. Crovella. Openness of the SIGCOMM conference, 2008. http:\/\/blog.sigcomm.org\/2008\/09\/openness_of_the_sigcomm_confer.html."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1435417.1435430"},{"key":"e_1_2_1_11_1","volume-title":"The principles of readability","author":"DuBay W. H.","year":"2004","unstructured":"W. H. DuBay . The principles of readability . 2004 . http:\/\/www.nald.ca\/library\/research\/readab\/readab.pdf. W. H. DuBay. The principles of readability. 2004. http:\/\/www.nald.ca\/library\/research\/readab\/readab.pdf."},{"key":"e_1_2_1_12_1","volume-title":"IEEE Global Internet Symposium Open Review Process, 2007","author":"Faloutsos M.","year":"2007","unstructured":"M. Faloutsos . IEEE Global Internet Symposium Open Review Process, 2007 . http:\/\/netsec.cs.uoregon.edu\/gi 2007 \/. M. Faloutsos. IEEE Global Internet Symposium Open Review Process, 2007. http:\/\/netsec.cs.uoregon.edu\/gi2007\/."},{"key":"e_1_2_1_13_1","volume-title":"Transmission of Information","author":"Fano R.","year":"1961","unstructured":"R. Fano . Transmission of Information . The MIT Press , Cambridge, MA , 1961 . R. Fano. Transmission of Information. The MIT Press, Cambridge, MA, 1961."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1070873.1070889"},{"key":"e_1_2_1_15_1","volume-title":"WOWCS","author":"Francis P.","year":"2008","unstructured":"P. Francis . Thoughts on improving review quality . In WOWCS , 2008 . P. Francis. Thoughts on improving review quality. In WOWCS, 2008."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"S. Keshav. July 2011 editor's message. SIGCOMM Comput. Commun. Rev. 41(3) 2011. S. Keshav. July 2011 editor's message. SIGCOMM Comput. Commun. Rev. 41(3) 2011.","DOI":"10.1145\/1925861.1925879"},{"key":"e_1_2_1_17_1","volume-title":"Naval Air Station Memphis","author":"Kincaid J. P.","year":"1975","unstructured":"J. P. Kincaid , R. P. Fishburne , R. L. Rogers , and B. S. Chissom . Derivation of new readability formulas. Technical report , Naval Air Station Memphis , 1975 . J. P. Kincaid, R. P. Fishburne, R. L. Rogers, and B. S. Chissom. Derivation of new readability formulas. Technical report, Naval Air Station Memphis, 1975."},{"key":"e_1_2_1_18_1","volume-title":"HotCRP Conference Management Software","author":"Kohler E.","year":"2012","unstructured":"E. Kohler . HotCRP Conference Management Software , 2012 . http:\/\/www.read.seas.harvard.edu\/~kohler\/hotcrp\/. E. Kohler. HotCRP Conference Management Software, 2012. http:\/\/www.read.seas.harvard.edu\/~kohler\/hotcrp\/."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1462571.1462581"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1967.10482916"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273445.1273456"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.2307\/2685263"},{"key":"e_1_2_1_23_1","volume-title":"EDAS Conference Management System","author":"Schulzrinne H.","year":"2012","unstructured":"H. Schulzrinne . EDAS Conference Management System , 2012 . http:\/\/edas.info\/doc\/. H. Schulzrinne. EDAS Conference Management System, 2012. http:\/\/edas.info\/doc\/."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001269.2001283"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1965724.1965728"}],"container-title":["ACM SIGCOMM Computer Communication Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2427036.2427040","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2427036.2427040","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:49:00Z","timestamp":1750236540000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2427036.2427040"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1,9]]},"references-count":25,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2013,1,9]]}},"alternative-id":["10.1145\/2427036.2427040"],"URL":"https:\/\/doi.org\/10.1145\/2427036.2427040","relation":{},"ISSN":["0146-4833"],"issn-type":[{"type":"print","value":"0146-4833"}],"subject":[],"published":{"date-parts":[[2012,1,9]]},"assertion":[{"value":"2012-01-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}