{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T15:13:43Z","timestamp":1768317223342,"version":"3.49.0"},"reference-count":94,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,11,1]],"date-time":"2019-11-01T00:00:00Z","timestamp":1572566400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMIS Database"],"published-print":{"date-parts":[[2019,11]]},"abstract":"<jats:p>In a recent critique of reviewers, Ralph (2016) stated that \"Peer review is prejudiced, capricious, inefficient, ineffective and generally unscientific\" (p. 274). Our research proposes that one way the peer review process could appear flawed is if those involved had different beliefs about what was important in evaluating research. We found evidence for a cognitive bias where respondents to a survey asking about the importance of particular validity and reliability method practices gave different answers depending on whether they were asked to answer the survey as a researcher or as a reviewer. Because researchers have higher motivation to publish research than reviewers do to review research, we theorize that motivational differences between researchers and reviewers leads to this bias and contributes to the perception that the review process is flawed. We discuss the implications of our findings for improving the peer review process in MIS.<\/jats:p>","DOI":"10.1145\/3371041.3371046","type":"journal-article","created":{"date-parts":[[2019,11,4]],"date-time":"2019-11-04T14:04:16Z","timestamp":1572876256000},"page":"52-70","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Cognitive Bias in the Peer Review Process"],"prefix":"10.1145","volume":"50","author":[{"given":"Chris","family":"Street","sequence":"first","affiliation":[{"name":"University of Regina, Regina, SK, Canada"}]},{"given":"Kerry W.","family":"Ward","sequence":"additional","affiliation":[{"name":"University of Nebraska, Omaha, Omaha, NE, USA"}]}],"member":"320","published-online":{"date-parts":[[2019,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2013.0476"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Aladwani A. M. (2013). A contingency model of citizens' attitudes toward e--government use. Electronic Government an International Journal 10(1) 68--81.  Aladwani A. M. (2013). A contingency model of citizens' attitudes toward e--government use. Electronic Government an International Journal 10(1) 68--81.","DOI":"10.1504\/EG.2013.051276"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"issue":"5","key":"e_1_2_1_4_1","doi-asserted-by":"crossref","first-page":"1219","DOI":"10.2307\/256856","article-title":"The review process and the fates of manuscripts submitted to AMJ","volume":"38","author":"Beyer J. M.","year":"1995","journal-title":"Academy of Management Journal"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23334"},{"issue":"2","key":"e_1_2_1_6_1","first-page":"23","article-title":"Scientific peer review: An analysis of the peer review process from the perspective of sociology of science theories","volume":"6","author":"Bornmann L.","year":"2008","journal-title":"Human Architecture: Journal of the Sociology of Self-Knowledge"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_2_1_8_1","volume-title":"H. Reis & C","author":"Brewer M.","year":"2000"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1468794106065006"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Chua C. E.H. Thatcher J. Niederman F. Chan Y.E. & Davidson E.J. (2018). ICIS Panel Report: Break your shackles! Emancipating information systems from the tyranny of peer review. Communications of the Association for Information Systems 43. https:\/\/doi.org\/10.17705\/1CAIS.04325  Chua C. E.H. Thatcher J. Niederman F. Chan Y.E. & Davidson E.J. (2018). ICIS Panel Report: Break your shackles! Emancipating information systems from the tyranny of peer review. Communications of the Association for Information Systems 43. https:\/\/doi.org\/10.17705\/1CAIS.04325","DOI":"10.17705\/1CAIS.04325"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5465\/amle.2013.0371"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.7302566"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-6924.2009.01109.x"},{"key":"e_1_2_1_14_1","unstructured":"Cooper D.R. & Schindler P. S. (2008). Business research methods 10th ed. Boston MA: McGraw Hill.  Cooper D.R. & Schindler P. S. (2008). Business research methods 10th ed. Boston MA: McGraw Hill."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-018-2664-3"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0040957"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1108\/09600031211225963"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/isj.12083"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jom.2004.12.001"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.3200\/GENP.136.3.231-242"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/2017204.2017209"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcrs.2017.08.002"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5465\/amr.1989.4308385"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.50.10.883"},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"Frey B. S. (2003). Publishing as prostitution?--Choosing between one's own ideas and academic success. Public Choice 116(1--2) 205--223.  Frey B. S. (2003). Publishing as prostitution?--Choosing between one's own ideas and academic success. Public Choice 116(1--2) 205--223.","DOI":"10.1023\/A:1024208701874"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biomaterials.2007.07.004"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1890\/0012-9623-95.4.431"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1080\/10511253.2017.1420809"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03812"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 25th European Conference on Information Systems (ECIS)","author":"Janze C.","year":"2017"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03817"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.280.3.240"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Kachewar S. G. & Smita B. S. (2013). Reviewer index: A new proposal of rewarding the reviewer. Mens sana monographs 11(1) 274--284.  Kachewar S. G. & Smita B. S. (2013). Reviewer index: A new proposal of rewarding the reviewer. Mens sana monographs 11(1) 274--284.","DOI":"10.4103\/0973-1229.109347"},{"key":"e_1_2_1_34_1","unstructured":"Kerlinger F. N. & Lee H. B. (2000). Foundations of behavioral research 4th ed. United States: Thompson Learning.  Kerlinger F. N. & Lee H. B. (2000). Foundations of behavioral research 4th ed. United States: Thompson Learning."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/0149206317743553"},{"issue":"1","key":"e_1_2_1_36_1","first-page":"13","article-title":"The case study research method: Overview and proposed guidelines for reporting and evaluation illustrated with health informatics case studies","volume":"2","author":"Kim J.","year":"2014","journal-title":"International Journal of Health Information Management Research"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Kirk J. & Miller M. L. (1986). Reliability and validity in qualitative research. Vol. 1 Sage.  Kirk J. & Miller M. L. (1986). Reliability and validity in qualitative research. Vol. 1 Sage.","DOI":"10.4135\/9781412985659"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/02701367.2014.898117"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the Americas Conference on Information Systems.","author":"Kohli R.","year":"2006"},{"issue":"3","key":"e_1_2_1_40_1","first-page":"176","article-title":"Qualitative research: Examining the misconceptions","volume":"22","author":"Kumar S. S.","year":"2015","journal-title":"South Asian Journal of Management"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.2307\/248698"},{"key":"e_1_2_1_42_1","unstructured":"Lee T. W. (1999). Using qualitative methods in organizational research. Thousand Oaks CA: Sage Publications.  Lee T. W. (1999). Using qualitative methods in organizational research. Thousand Oaks CA: Sage Publications."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.22784"},{"key":"e_1_2_1_44_1","unstructured":"Liao H. (2015). Reporting credibility in educational evaluation studies that use qualitative methods: A mixed methods research synthesis (Unpublished doctoral dissertation). Patton College of Education Ohio University.  Liao H. (2015). Reporting credibility in educational evaluation studies that use qualitative methods: A mixed methods research synthesis (Unpublished doctoral dissertation). Patton College of Education Ohio University."},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Lincoln Y.S. & Guba E.G. (1985). Naturalistic Inquiry. Newbury Park CA: Sage Publications.  Lincoln Y.S. & Guba E.G. (1985). Naturalistic Inquiry. Newbury Park CA: Sage Publications.","DOI":"10.1016\/0147-1767(85)90062-8"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1177\/1056492614554773"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01173636"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2008.04.005"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Marques K. C. M. Camacho R. R. & Alcantara C. C. V. (2015). Assessment of the methodological rigor of case studies in the field of management accounting. Revista Contabilidade & Finan\u00e7as 26(67) 27--42.  Marques K. C. M. Camacho R. R. & Alcantara C. C. V. (2015). Assessment of the methodological rigor of case studies in the field of management accounting. Revista Contabilidade & Finan\u00e7as 26(67) 27--42.","DOI":"10.1590\/1808-057x201500280"},{"key":"e_1_2_1_50_1","author":"Matthews D.","year":"2016","journal-title":"Journal to rank peer reviewers to improve response times. Retrieved on"},{"key":"e_1_2_1_51_1","unstructured":"McDonald J.H. (2014). Handbook of biological statistics (3rd ed.). Baltimore MD: Sparky House Publishing.  McDonald J.H. (2014). Handbook of biological statistics (3rd ed.). Baltimore MD: Sparky House Publishing."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1086\/598847"},{"key":"e_1_2_1_53_1","unstructured":"Merriam S. B. (1998). Qualitative research and case study applications in education. San Francisco CA: Jossey-Bass Inc.  Merriam S. B. (1998). Qualitative research and case study applications in education. San Francisco CA: Jossey-Bass Inc."},{"key":"e_1_2_1_54_1","unstructured":"Mitchell T. R. Beach L. R. & Smith K. G. (1985). Some data on publishing from the authors' and reviewers' perspectives. Publishing in the organizational sciences 248--264.  Mitchell T. R. Beach L. R. & Smith K. G. (1985). Some data on publishing from the authors' and reviewers' perspectives. Publishing in the organizational sciences 248--264."},{"key":"e_1_2_1_55_1","unstructured":"Miles M. B. & Huberman A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Newbury Park CA: Sage Publications.  Miles M. B. & Huberman A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Newbury Park CA: Sage Publications."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1108\/MRR-03-2012-0049"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.5465\/amj.2006.21794661"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03814"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.22798"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.paid.2013.02.018"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1057\/ejis.2010.49"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.1100.0607"},{"key":"e_1_2_1_63_1","doi-asserted-by":"crossref","unstructured":"Payne G. & Payne J. (2004). Positivism and realism: Key concepts in social research Sage Publications Newbury Park CA.  Payne G. & Payne J. (2004). Positivism and realism: Key concepts in social research Sage Publications Newbury Park CA.","DOI":"10.4135\/9781849209397"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00011183"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.04229"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.respol.2012.03.005"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094428108319905"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1177\/1056492617726273"},{"issue":"1","key":"e_1_2_1_69_1","first-page":"1","article-title":"Editor's comments: Developing our authors","volume":"40","author":"Ragins B. R.","year":"2015","journal-title":"Academy of Management"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03813"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03818"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1093\/brain\/123.9.1964"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797613511824"},{"key":"e_1_2_1_74_1","doi-asserted-by":"crossref","unstructured":"Saunders C. (2005a). Looking for diamond cutters. MIS Quarterly 29(1) iii-viii.  Saunders C. (2005a). Looking for diamond cutters. MIS Quarterly 29(1) iii-viii.","DOI":"10.2307\/25148664"},{"key":"e_1_2_1_75_1","doi-asserted-by":"crossref","unstructured":"Saunders C. (2005b). From the trenches: Thoughts on developmental reviewing. MIS Quarterly 29(2) iii-xii  Saunders C. (2005b). From the trenches: Thoughts on developmental reviewing. MIS Quarterly 29(2) iii-xii","DOI":"10.2307\/25148675"},{"key":"e_1_2_1_76_1","volume-title":"International Conference on Intelligent Technologies for Interactive Entertainment. Springer, Cham.","author":"Siebelink J.","year":"2016"},{"issue":"1","key":"e_1_2_1_77_1","first-page":"27","article-title":"Peer reviewer non-performance: The need to replenish the \"commons","volume":"42","author":"Sipior J. C.","year":"2018","journal-title":"Communications of the Association for Information Systems"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1177\/014107680609900414"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03816"},{"key":"e_1_2_1_80_1","doi-asserted-by":"crossref","unstructured":"Stafford T. F. (2018a). Reviews reviewers and reviewing: The \"tragedy of the commons\" in the scientific publication process. Communications of the Association for Information Systems 42(1) Article 25.  Stafford T. F. (2018a). Reviews reviewers and reviewing: The \"tragedy of the commons\" in the scientific publication process. Communications of the Association for Information Systems 42(1) Article 25.","DOI":"10.17705\/1CAIS.04225"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.04230"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1177\/1056492603258972"},{"key":"e_1_2_1_83_1","doi-asserted-by":"crossref","unstructured":"Straub D. W. (2009). Diamond mining or coal mining? Which reviewing industry are we in? MIS Quarterly 33(2) iii-viii.  Straub D. W. (2009). Diamond mining or coal mining? Which reviewing industry are we in? MIS Quarterly 33(2) iii-viii.","DOI":"10.2307\/20650289"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1177\/1938965511419844"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.respol.2012.04.014"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094428108318429"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1707323114"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1559-1816.1999.tb00125.x"},{"key":"e_1_2_1_89_1","doi-asserted-by":"crossref","unstructured":"Venkatesh V. Brown S.A. & Bala H. (2013). Bridging the qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS Quarterly 37(1).  Venkatesh V. Brown S.A. & Bala H. (2013). Bridging the qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS Quarterly 37(1).","DOI":"10.25300\/MISQ\/2013\/37.1.02"},{"key":"e_1_2_1_90_1","unstructured":"Vogt W. P. (1999). Dictionary of Statistics and Methodology (2nd ed.). Thousand Oaks CA Sage Publications Inc.  Vogt W. P. (1999). Dictionary of Statistics and Methodology (2nd ed.). Thousand Oaks CA Sage Publications Inc."},{"key":"e_1_2_1_91_1","doi-asserted-by":"crossref","unstructured":"Weber R. (1999). The journal review process: A manifesto for change. Communications of the AIS 2(12).  Weber R. (1999). The journal review process: A manifesto for change. Communications of the AIS 2(12).","DOI":"10.17705\/1CAIS.00212"},{"key":"e_1_2_1_92_1","unstructured":"Weller A. C. (2001). Editorial peer review: Its strengths and weaknesses. Information Today Inc.  Weller A. C. (2001). Editorial peer review: Its strengths and weaknesses. Information Today Inc."},{"key":"e_1_2_1_93_1","unstructured":"Yin R.K. (1994). Case Study Research: Design and Methods (5th ed.). Thousand Oaks CA Sage Publications Inc.  Yin R.K. (1994). Case Study Research: Design and Methods (5th ed.). Thousand Oaks CA Sage Publications Inc."},{"key":"e_1_2_1_94_1","doi-asserted-by":"crossref","unstructured":"Zavattaro S. M. & Orr S. K. (2017). Developing a research agenda: Your bread and butter (pp. 95--120). Reflections on Academic Lives. New York: Palgrave Macmillan.  Zavattaro S. M. & Orr S. K. (2017). Developing a research agenda: Your bread and butter (pp. 95--120). Reflections on Academic Lives. New York: Palgrave Macmillan.","DOI":"10.1057\/978-1-137-60009-7_5"}],"container-title":["ACM SIGMIS Database: the DATABASE for Advances in Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3371041.3371046","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3371041.3371046","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:23:36Z","timestamp":1750202616000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3371041.3371046"}},"subtitle":["Understanding a Source of Friction between Reviewers and Researchers"],"short-title":[],"issued":{"date-parts":[[2019,11]]},"references-count":94,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,11]]}},"alternative-id":["10.1145\/3371041.3371046"],"URL":"https:\/\/doi.org\/10.1145\/3371041.3371046","relation":{},"ISSN":["0095-0033","1532-0936"],"issn-type":[{"value":"0095-0033","type":"print"},{"value":"1532-0936","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11]]},"assertion":[{"value":"2019-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}