{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T15:05:47Z","timestamp":1776092747399,"version":"3.50.1"},"reference-count":53,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[2011,4,26]],"date-time":"2011-04-26T00:00:00Z","timestamp":1303776000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2011,4,26]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>This paper seeks to discuss reliability problems associated with questionnaires, commonly employed in library and information science. It aims to focus on the effects of \u201ccommon method variance\u201d (CMV), which is a form of bias, and ways of countering these effects.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>The paper critically reviews the use of existing tools for demonstrating reliability in questionnaire\u2010based studies. In particular, it focuses on Cronbach's alpha, \u201cHarman's single factor test\u201d and Lindell and Whitney's \u201cmarker variable\u201d approach. The paper introduces an illustrative case study based on the work on metacognition and web searching. It goes on to make recommendations for researchers considering using a questionnaire\u2010based approach.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>CMV is a problem affecting questionnaire\u2010based studies in different disciplines across social and information science. Where questionnaire items are more abstract, CMV has been found to be more of a problem. The widely used Cronbach alpha measure, of the reliability of a questionnaire, may often be affected by CMV. Where method bias dominates the results, a high alpha score may do no more than indicate that the questionnaire consistently allows participants to accede to their biases. \u201cHarman's single factor test\u201d is criticised as lacking sufficient foundation, and it is concluded that it should not be used. The marker variable approach is a useful option but must be applied with caution.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Practical implications<\/jats:title><jats:p>A number of practical implications for researchers are drawn. Questionnaire\u2010based work investigating abstract constructs should be assumed to carry a high likelihood of CMV, and therefore should attempt to avoid it and demonstrate the degree of success in this regard. Otherwise, interpretation of the results should assume CMV. A number of approaches to assessing and isolating CMV are discussed.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>The paper draws attention to a problem that is arguably often overlooked in questionnaire\u2010based studies, namely method bias. It discusses a number of approaches whereby it may be identified and controlled.<\/jats:p><\/jats:sec>","DOI":"10.1108\/00220411111124569","type":"journal-article","created":{"date-parts":[[2011,5,21]],"date-time":"2011-05-21T07:05:52Z","timestamp":1305961552000},"page":"507-524","source":"Crossref","is-referenced-by-count":88,"title":["Countering method bias in questionnaire\u2010based user studies"],"prefix":"10.1108","volume":"67","author":[{"given":"Genevieve","family":"Gorrell","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nigel","family":"Ford","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrew","family":"Madden","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Holdridge","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Barry","family":"Eaglestone","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2022021219555939300_b1","doi-asserted-by":"crossref","unstructured":"Alexander, P.A. and Judy, J.E. (1988), \u201cThe interaction of domain\u2010specific and strategic knowledge in academic performance\u201d, Review of Educational Research, Vol. 58 No. 4, pp. 375\u2010404.","DOI":"10.3102\/00346543058004375"},{"key":"key2022021219555939300_b2","doi-asserted-by":"crossref","unstructured":"Bernerth, J.B., Armenakis, A.A., Feild, H.S., Giles, W.F. and Walker, H.J. (2008), \u201cThe influence of personality differences between subordinates and supervisors on perceptions of LMX: an empirical investigation\u201d, Group Organization Management, Vol. 33 No. 2, pp. 216\u201040.","DOI":"10.1177\/1059601106293858"},{"key":"key2022021219555939300_b3","doi-asserted-by":"crossref","unstructured":"Brewer, M.B., Campbell, D.T. and Crano, W.D. (1970), \u201cTesting a single factor model as an alternative to the misuse of partial correlations in hypothesis\u2010testing research\u201d, Sociometry, Vol. 33, pp. 1\u20108.","DOI":"10.2307\/2786268"},{"key":"key2022021219555939300_b5","unstructured":"Brown, A. (1987), \u201cMetacognition, executive control, self\u2010regulation, and other more mysterious mechanisms\u201d, Metacognition, Motivation, and Understanding, Lawrence Earlbaum Associates, Hillsdale, NJ."},{"key":"key2022021219555939300_b4","unstructured":"Brown, A.L., Bransford, J.D., Ferrara, R.A. and Campione, J.C. (1983), \u201cLearning, remembering, and understanding\u201d, in Flavell, J.H. and Markman, E.M. (Eds), Handbook of Child Psychology, 3, Cognitive Development, 4th ed., Wiley, New York, NY, pp. 77\u2010166."},{"key":"key2022021219555939300_b6","doi-asserted-by":"crossref","unstructured":"Bstieler, L. and Hemmert, M. (2008), \u201cInfluence of tie strength and behavioural factors on effective knowledge acquisition: a study of Korean new product alliances\u201d, Asian Business and Management, Vol. 7, pp. 75\u201094.","DOI":"10.1057\/palgrave.abm.9200245"},{"key":"key2022021219555939300_b7","doi-asserted-by":"crossref","unstructured":"Campbell, D.T. and Fiske, D.W. (1959), \u201cConvergent and discriminant validation by the multitrait\u2010multimethod matrix\u201d, Psychological Bulletin, Vol. 58, pp. 56\u201081.","DOI":"10.1037\/h0046016"},{"key":"key2022021219555939300_b8","doi-asserted-by":"crossref","unstructured":"Carr, A.S. and Kaynak, H. (2007), \u201cCommunication methods, information sharing, supplier development and performance\u201d, International Journal of Operations & Production Management, Vol. 27 No. 4, pp. 346\u201070.","DOI":"10.1108\/01443570710736958"},{"key":"key2022021219555939300_b9","doi-asserted-by":"crossref","unstructured":"Carr, A. and Muthusamy, S.K. (2008), \u201cThe cross\u2010functional coordination between operations, marketing, purchasing and engineering and the impact on performance\u201d, International Journal of Manufacturing Technology and Management, Vol. 13 No. 1, pp. 55\u201077.","DOI":"10.1504\/IJMTM.2008.015973"},{"key":"key2022021219555939300_b10","unstructured":"\u00c7etinkaya, P. and Ertkin, E. (2002), \u201cAssessment of metacognition and its relationship with reading comprehension, achievement, and aptitude\u201d, Bo\u011fazi\u00e7i University Journal of Education, Vol. 19 No. 1, pp. 1\u201011."},{"key":"key2022021219555939300_b11","doi-asserted-by":"crossref","unstructured":"Chi, L., Jones, K.G., Lederer, A.L., Li, P., Newkirk, H.E. and Sethi, V. (2004), \u201cEnvironmental assessment in strategic information systems planning\u201d, International Journal of Information Management, Vol. 25 No. 3, pp. 253\u201069.","DOI":"10.1016\/j.ijinfomgt.2004.12.004"},{"key":"key2022021219555939300_b12","doi-asserted-by":"crossref","unstructured":"Chungtai, A.A. (2008), \u201cImpact of job involvement on in\u2010role job performance and organizational citizenship behaviour\u201d, Journal of the Institute of Behavioral and Applied Management, Vol. 9 No. 2, pp. 169\u201083.","DOI":"10.21818\/001c.17123"},{"key":"key2022021219555939300_b13","doi-asserted-by":"crossref","unstructured":"Cote, J.A. and Buckley, M.R. (1987), \u201cEstimating trait, method, and error variance: generalizing across 70 construct validation studies\u201d, Journal of Marketing Research, Vol. 24 No. 3, pp. 315\u201018.","DOI":"10.1177\/002224378702400308"},{"key":"key2022021219555939300_b14","doi-asserted-by":"crossref","unstructured":"Crampton, S.A. and Wagner, J.A. (1994), \u201cPercept\u2010percept inflation in micro\u2010organizational research: an investigation of prevalence and effect\u201d, Journal of Applied Psychology, Vol. 79, pp. 67\u201076.","DOI":"10.1037\/0021-9010.79.1.67"},{"key":"key2022021219555939300_b15","doi-asserted-by":"crossref","unstructured":"Cronbach, L.J. (1951), \u201cCoefficient alpha and the internal structure of tests\u201d, Psychometrika, Vol. 16 No. 3, pp. 297\u2010334.","DOI":"10.1007\/BF02310555"},{"key":"key2022021219555939300_b16","doi-asserted-by":"crossref","unstructured":"Darnall, N., Jolley, G.J. and Handfield, R. (2008), \u201cEnvironmental management systems and green supply chain management: complements for sustainability?\u201d, Business Strategy and the Environment, Vol. 18, pp. 30\u201045.","DOI":"10.1002\/bse.557"},{"key":"key2022021219555939300_b17","doi-asserted-by":"crossref","unstructured":"De Cuyper, N., Bernhard\u2010Oettel, C., Berntson, E., De Witte, H. and Alarco, B. (2008), \u201cEmployability and employees' wellbeing: mediation by job insecurity\u201d, Applied Psychology, Vol. 57 No. 3, pp. 488\u2010509.","DOI":"10.1111\/j.1464-0597.2008.00332.x"},{"key":"key2022021219555939300_b18","doi-asserted-by":"crossref","unstructured":"Doty, D.H. and Glick, W.H. (1998), \u201cCommon methods bias: does common methods variance really bias results?\u201d, Organizational Research Methods, Vol. 1, pp. 374\u2010406.","DOI":"10.1177\/109442819814002"},{"key":"key2022021219555939300_b19","doi-asserted-by":"crossref","unstructured":"Flavell, J. (1979), \u201cMetacognition and cognitive monitoring: a new area of cognitive\u2010developmental inquiry\u201d, American Psychologist, Vol. 34, pp. 906\u201011.","DOI":"10.1037\/0003-066X.34.10.906"},{"key":"key2022021219555939300_b20","doi-asserted-by":"crossref","unstructured":"Ford, N. (2004), \u201cModeling cognitive processes in information seeking: from Popper to Pask\u201d, Journal of the American Society for Information Science and Technology, Vol. 55 No. 9, pp. 769\u201082.","DOI":"10.1002\/asi.20021"},{"key":"key2022021219555939300_b22","unstructured":"Gorrell, G., Eaglestone, B., Ford, N., Holdridge, P. and Madden, A. (2008), \u201cThe role of meta\u2010cognition in web searching to support inquiry\u2010based learning\u201d, Proceedings of the ECEL, 2008, pp. 443\u201050."},{"key":"key2022021219555939300_b21","doi-asserted-by":"crossref","unstructured":"Gorrell, G., Madden, A., Holdridge, P., Ford, N. and Eaglestone, B. (2009), \u201cTowards \u2018metacognitively aware\u2019 IR systems: an initial user study\u201d, Journal of Documentation, Vol. 65 No. 3, pp. 446\u201069.","DOI":"10.1108\/00220410910952429"},{"key":"key2022021219555939300_b23","doi-asserted-by":"crossref","unstructured":"Greene, C.N. and Organ, D.W. (1973), \u201cAn evaluation of causal models linking the received role with job satisfaction\u201d, Administrative Science Quarterly, Vol. 18 No. 1, pp. 95\u2010103.","DOI":"10.2307\/2391931"},{"key":"key2022021219555939300_b24","unstructured":"Hannafin, M.J., Land, S. and Oliver, K. (1999), \u201cOpen learning environments: foundations, methods, and models\u201d, in Reigeluth, C.M. (Ed.), Instructional\u2010Design Theories and Models: A New Paradigm of Instruction Theory, Erlbaum, Hillsdale, NJ, pp. 115\u201040."},{"key":"key2022021219555939300_b25","unstructured":"Harman, H.H. (1967), Modern Factor Analysis, University of Chicago Press, Chicago, IL."},{"key":"key2022021219555939300_b26","doi-asserted-by":"crossref","unstructured":"Hill, J.R. and Hannafin, M.J. (1997), \u201cCognitive strategies and learning from the world wide web\u201d, Educational Technology Research and Development, Vol. 45 No. 4, pp. 37\u201064.","DOI":"10.1007\/BF02299682"},{"key":"key2022021219555939300_b27","doi-asserted-by":"crossref","unstructured":"Jimmieson, N.L., Peach, M. and White, K.M. (2008), \u201cUtilizing the theory of planned behaviour to inform change management: an investigation of employee intentions to support organizational change\u201d, Journal of Applied Behavioural Science, Vol. 44 No. 2, pp. 237\u201062.","DOI":"10.1177\/0021886307312773"},{"key":"key2022021219555939300_b28","doi-asserted-by":"crossref","unstructured":"Karatepe, O.M. and Magaji, A. (2008), \u201cWork\u2010family conflict and facilitation in the hotel industry: a study in Nigeria\u201d, Cornell Hospitality Quarterly, Vol. 49 No. 4, pp. 395\u2010412.","DOI":"10.1177\/1938965508326282"},{"key":"key2022021219555939300_b29","doi-asserted-by":"crossref","unstructured":"Kemery, E.R. and Dunlap, W.P. (1986), \u201cPartialling factor scores does not control method variance: a reply to Podsakoff and Todor\u201d, Journal of Management, Vol. 12 No. 4, pp. 525\u201030.","DOI":"10.1177\/014920638601200407"},{"key":"key2022021219555939300_b30","doi-asserted-by":"crossref","unstructured":"Kim, Y.J., Song, J. and Koo, C. (2008), \u201cExploring the effect of strategic positioning on firm performance in the e\u2010business context\u201d, Journal of Information Management, Vol. 28 No. 3, pp. 203\u201014.","DOI":"10.1016\/j.ijinfomgt.2008.02.004"},{"key":"key2022021219555939300_b31","unstructured":"Kourteli, L. (2005), \u201cScanning the business external environment for information: evidence from Greece\u201d, Information Research, Vol. 11 No. 1."},{"key":"key2022021219555939300_b32","doi-asserted-by":"crossref","unstructured":"Lau, A.Y.S. and Coiera, E.W. (2008), \u201cImpact of web searching and social feedback on consumer decision making: a prospective online experiment\u201d, Journal of Medical Internet Research, Vol. 10 No. 1, p. e2.","DOI":"10.2196\/jmir.963"},{"key":"key2022021219555939300_b33","doi-asserted-by":"crossref","unstructured":"Lindell, M.K. and Whitney, D.J. (2001), \u201cAccounting for common method variance in cross\u2010sectional research designs\u201d, Journal of Applied Psychology, Vol. 86 No. 1, pp. 114\u201021.","DOI":"10.1037\/0021-9010.86.1.114"},{"key":"key2022021219555939300_b34","doi-asserted-by":"crossref","unstructured":"Malhotra, N.K., Kim, S. and Patil, A. (2006), \u201cCommon method variance in IS research: a comparison of alternative approaches and a reanalysis of past research\u201d, Management Science, Vol. 52 No. 12, pp. 1865\u201083.","DOI":"10.1287\/mnsc.1060.0597"},{"key":"key2022021219555939300_b35","doi-asserted-by":"crossref","unstructured":"Mansourian, Y. (2008), \u201cContextual elements and conceptual components of information visibility on the web\u201d, Library Hi Tech, Vol. 26 No. 3, pp. 440\u201053.","DOI":"10.1108\/07378830810903355"},{"key":"key2022021219555939300_b36","doi-asserted-by":"crossref","unstructured":"Metcalfe, J. and Shimamura, A.P. (1994), Metacognition: Knowing about Knowing, MIT Press, Cambridge, MA.","DOI":"10.7551\/mitpress\/4561.001.0001"},{"key":"key2022021219555939300_b37","doi-asserted-by":"crossref","unstructured":"Muis, K.R., Winne, P.H. and Jamieson\u2010Noel, D. (2007), \u201cUsing a multitrait\u2010multimethod analysis to examine conceptual similarities of three self\u2010regulated learning inventories\u201d, British Journal of Educational Psychology, Vol. 77 No. 1, pp. 177\u201095.","DOI":"10.1348\/000709905X90876"},{"key":"key2022021219555939300_b38","doi-asserted-by":"crossref","unstructured":"Newkirk, H.E., Lederer, A.L. and Johnson, A.M. (2008), \u201cRapid business and IT change: drivers for strategic information systems planning?\u201d, European Journal of Information Systems, Vol. 17, pp. 198\u2010218.","DOI":"10.1057\/ejis.2008.16"},{"key":"key2022021219555939300_b39","doi-asserted-by":"crossref","unstructured":"O'Neil, H.F. and Abedi, J. (1996), Reliability and Validity of a State Metacognitive Inventory: Potential for Alternative Assessment, CSE Tech. Rep. 469, National Center for Research on Evaluation, Standards, and Student Testing (CRESST), University of California, Los Angeles, CA.","DOI":"10.1037\/e650722011-001"},{"key":"key2022021219555939300_b40","doi-asserted-by":"crossref","unstructured":"Paulraj, A., Chen, I.J. and Flynn, J. (2006), \u201cLevels of strategic purchasing: impact on supply integration and performance\u201d, Journal of Purchasing and Supply Management, Vol. 12 No. 3, pp. 107\u201022.","DOI":"10.1016\/j.pursup.2006.08.002"},{"key":"key2022021219555939300_b42","doi-asserted-by":"crossref","unstructured":"Podsakoff, P.M. and Organ, D.W. (1986), \u201cSelf\u2010reports in organizational research: problems and prospects\u201d, Journal of Management, Vol. 12 No. 2, pp. 531\u201044.","DOI":"10.1177\/014920638601200408"},{"key":"key2022021219555939300_b44","doi-asserted-by":"crossref","unstructured":"Podsakoff, P.M. and Todor, W.D. (1985), \u201cRelationships between leader reward and punishment behavior and group processes and productivity\u201d, Journal of Management, Vol. 11 No. 1, pp. 55\u201073.","DOI":"10.1177\/014920638501100106"},{"key":"key2022021219555939300_b41","doi-asserted-by":"crossref","unstructured":"Podsakoff, P.M., MacKenzie, S.B., Lee, J.Y. and Podsakoff, N.P. (2003), \u201cCommon method biases in behavioral research: a critical review of the literature and recommended remedies\u201d, Journal of Applied Psychology, Vol. 88 No. 5, pp. 879\u2010903.","DOI":"10.1037\/0021-9010.88.5.879"},{"key":"key2022021219555939300_b43","doi-asserted-by":"crossref","unstructured":"Podsakoff, P.M., Todor, W.D., Grover, R.A. and Huber, V.L. (1984), \u201cSituational moderators of leader reward and punishment behaviors: fact or fiction?\u201d, Organizational Behavior and Human Performance, Vol. 34, pp. 21\u201063.","DOI":"10.1016\/0030-5073(84)90036-9"},{"key":"key2022021219555939300_b45","doi-asserted-by":"crossref","unstructured":"Rego, A., Sousa, F., Pina e Cunha, M., Correia, A. and Saur\u2010Amarai, I. (2007), \u201cLeader self\u2010reported emotional intelligence and perceived employee creativity: an exploratory study\u201d, Creativity and Innovation Management, Vol. 16 No. 3, pp. 250\u201064.","DOI":"10.1111\/j.1467-8691.2007.00435.x"},{"key":"key2022021219555939300_b46","doi-asserted-by":"crossref","unstructured":"Schraw, G. and Dennison, R.S. (1994), \u201cAssessing metacognitive awareness\u201d, Contemporary Educational Psychology, Vol. 19, pp. 460\u201075.","DOI":"10.1006\/ceps.1994.1033"},{"key":"key2022021219555939300_b47","doi-asserted-by":"crossref","unstructured":"Schriesheim, C.A. (1979), \u201cThe similarity of individual\u2010directed and group\u2010directed leader behavior descriptions\u201d, Academy of Management Journal, Vol. 22 No. 2, pp. 345\u201055.","DOI":"10.5465\/255594"},{"key":"key2022021219555939300_b48","doi-asserted-by":"crossref","unstructured":"Schriesheim, J.F. (1980), \u201cThe social context of leader\u2010subordinate relations: an investigation of the effects of group cohesiveness\u201d, Journal of Applied Psychology, Vol. 65 No. 2, pp. 183\u201094.","DOI":"10.1037\/0021-9010.65.2.183"},{"key":"key2022021219555939300_b49","doi-asserted-by":"crossref","unstructured":"Song, M., Dyer, B. and Thieme, R.J. (2006), \u201cConflict management and innovation performance: an integrated contingency perspective\u201d, Journal of the Academy of Marketing Science, Vol. 34 No. 3, pp. 341\u201056.","DOI":"10.1177\/0092070306286705"},{"key":"key2022021219555939300_b50","doi-asserted-by":"crossref","unstructured":"Tang, T.L.P., Sutarso, T., Akande, A. and Allen, M.W. (2006), \u201cThe love of money and pay level satisfaction: measurement and functional equivalence in 29 geopolitical entities around the world\u201d, Management and Organizational Review, Vol. 2 No. 3, pp. 423\u201052.","DOI":"10.1111\/j.1740-8784.2006.00051.x"},{"key":"key2022021219555939300_b51","doi-asserted-by":"crossref","unstructured":"Thacker, R.A. and Wayne, S.J. (1995), \u201cAn examination of the relationship between upward influence tactics and assessments of promotability\u201d, Journal of Management, Vol. 21 No. 4, pp. 739\u201056.","DOI":"10.1177\/014920639502100408"},{"key":"key2022021219555939300_b52","doi-asserted-by":"crossref","unstructured":"Woszczynski, A.B. and Whitman, M.E. (2004), \u201cThe problem of common method variance in IS research\u201d, in Woszczynski, A.B. and Whitman, M.E. (Eds), The Handbook of Information Systems Research, Idea Group, Hershey, PA, pp. 66\u201077.","DOI":"10.4018\/978-1-59140-144-5.ch005"},{"key":"key2022021219555939300_b53","doi-asserted-by":"crossref","unstructured":"Zhang, X. and Chen, R. (2008), \u201cExamining the mechanism of the value co\u2010creation with customers\u201d, International Journal of Production Economics, Vol. 116 No. 2, pp. 242\u201050.","DOI":"10.1016\/j.ijpe.2008.09.004"}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/00220411111124569","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220411111124569\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220411111124569\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T23:38:21Z","timestamp":1753400301000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/67\/3\/507-524\/197925"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,4,26]]},"references-count":53,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2011,4,26]]}},"alternative-id":["10.1108\/00220411111124569"],"URL":"https:\/\/doi.org\/10.1108\/00220411111124569","relation":{},"ISSN":["0022-0418"],"issn-type":[{"value":"0022-0418","type":"print"}],"subject":[],"published":{"date-parts":[[2011,4,26]]}}}