{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T07:24:43Z","timestamp":1771053883837,"version":"3.50.1"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T00:00:00Z","timestamp":1714348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMIS Database"],"published-print":{"date-parts":[[2024,4,29]]},"abstract":"<jats:p>The abundance of data available to researchers has led to increasing interest in data-derived theoretical development. Although this is a valid method of deriving theoretical models, it is subject to numerous limitations and hazards that may threaten the validity and usefulness of the models. The purpose of this paper is to critique empirically driven theoretical development. Our goal is to offer a cautionary tale about the limits of derivation of theory from empirical analysis in the hopes that our analysis and critique can strengthen empirical derivation of theory. In this paper, we use the empirical derivation of the Unified Model of Information Security Policy Compliance (UMISPC) as a research case study to illustrate some of these limitations and risks. For example, we critique the opportunistic dropping of theoretical paths based on statistical results, cautioning that doing so is insufficient for forming new theory. We also report several attempts at validating UMISPC through replication, including our own, which used data from a survey of 525 employed American adults. Comparison of the replications and original model indicates a general failure to replicate substantial portions of the original paper. We discuss five specific pitfalls associated with empirically driven model development and make recommendations for future studies that use inductive, data-driven approaches to derive theoretical models.<\/jats:p>","DOI":"10.1145\/3663682.3663689","type":"journal-article","created":{"date-parts":[[2024,5,3]],"date-time":"2024-05-03T10:06:07Z","timestamp":1714730767000},"page":"120-146","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["The Limits of Empiricism: A Critique of Data-Driven Theory Development"],"prefix":"10.1145","volume":"55","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3924-1859","authenticated-orcid":false,"given":"Craig","family":"Van Slyke","sequence":"first","affiliation":[{"name":"Computer Information Systems, Louisiana Tech University, Ruston, LA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4222-9730","authenticated-orcid":false,"given":"Arnold","family":"Kamis","sequence":"additional","affiliation":[{"name":"International Business School, Brandeis University, Waltham, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,3]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-021-01665-8"},{"key":"e_1_2_1_2_1","volume-title":"The micro-task market for lemons: Data quality on Amazon's Mechanical Turk. Political Science Research and Methods, 1--20","author":"Ahler D. J.","year":"2019","unstructured":"Ahler, D. J., Roush, C. E., & Sood, G. (2019). The micro-task market for lemons: Data quality on Amazon's Mechanical Turk. Political Science Research and Methods, 1--20."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222220307"},{"key":"e_1_2_1_4_1","volume-title":"The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine, 16(7)","author":"Anderson C.","year":"2008","unstructured":"Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine, 16(7)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1037\/14805-038"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327663JCP0902_4"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.2307\/41409971"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2018.0774"},{"key":"e_1_2_1_9_1","article-title":"Meta-analysis of the unified theory of acceptance and use of technology (UTAUT): Challenging its validity and charting a research agenda in the red ocean","author":"Blut M.","year":"2021","unstructured":"Blut, M., Chong, A., Tsiga, Z., & Venkatesh, V. (2021). Meta-analysis of the unified theory of acceptance and use of technology (UTAUT): Challenging its validity and charting a research agenda in the red ocean. Journal of the Association for Information Systems, forthcoming.","journal-title":"Journal of the Association for Information Systems, forthcoming."},{"key":"e_1_2_1_10_1","volume-title":"The promise and peril of big data","author":"Bollier D.","year":"2010","unstructured":"Bollier, D., & Firestone, C. M. (2010). The promise and peril of big data. Aspen Institute, Communications and Society Program, Washington, DC."},{"issue":"1","key":"e_1_2_1_11_1","first-page":"301","article-title":"Next-generation information systems theorizing: A call to action","volume":"45","author":"Burton-Jones A.","year":"2021","unstructured":"Burton-Jones, A., Butler, B. S., Scott, S., & Xu, S. X. (2021). Next-generation information systems theorizing: A call to action. Management Information Systems Quarterly, 45(1), 301--314.","journal-title":"Management Information Systems Quarterly"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2017.0704"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2017.0702"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaf0918"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-018-0399-z"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/1558689807299526"},{"key":"e_1_2_1_17_1","volume-title":"Big data need big theory too. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2080)","author":"Coveney P. V.","year":"2016","unstructured":"Coveney, P. V., Dougherty, E. R., & Highfield, R. R. (2016). Big data need big theory too. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2080)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2019\/15117"},{"key":"e_1_2_1_19_1","volume-title":"Ancient and medieval empiricism. Stanford Encyclopedia of Philosophy. https:\/\/plato.stanford.edu\/entries\/empiricism-ancient-medieval\/","author":"Dawes G. W.","year":"2017","unstructured":"Dawes, G. W. (2017). Ancient and medieval empiricism. Stanford Encyclopedia of Philosophy. https:\/\/plato.stanford.edu\/entries\/empiricism-ancient-medieval\/"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0039191"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082--989X.4.3.272"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523813"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1097\/EDE.0b013e31827886f7"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2020\/15685"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-018-1257-z"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/08824096.2016.1214121"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1053\/j.seminhematol.2008.04"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.2307\/25148742"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1080\/1097198X.2019.1701623"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2015\/39.2.01"},{"key":"e_1_2_1_31_1","volume-title":"Multivariate data analysis","author":"Hair J. F.","year":"2019","unstructured":"Hair, J. F., Black, B., Babin, B., Anderson, R. E., & Tatham, R. L. (2019). Multivariate data analysis (8th ed.). Cengage.","edition":"8"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.3288"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1177\/1476127007077559"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.2307\/4132315"},{"key":"e_1_2_1_35_1","volume-title":"What's wrong with ICANN and how to fix it. ICANN Watch","author":"Johnson D. R.","year":"2011","unstructured":"Johnson, D. R., & Crawford, S. P. (2011). What's wrong with ICANN and how to fix it. ICANN Watch. http:\/\/www.icannwatch.org\/archive\/whats_wrong_with_icann.htm"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infoandorg.2019.01.001"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11135-017-0584-6"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/1473871611415994"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.2307\/249133"},{"key":"e_1_2_1_40_1","volume-title":"Theory building with big data-driven research--Moving away from the ?what\" towards the \"why.\" International Journal of Information Management, 54","author":"Kar A. K.","year":"2020","unstructured":"Kar, A. K., & Dwivedi, Y. K. (2020). Theory building with big data-driven research--Moving away from the ?what\" towards the \"why.\" International Journal of Information Management, 54."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382577.2382579"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/00913367.2016.1269304"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629175.1629210"},{"key":"e_1_2_1_44_1","article-title":"Data from investigating variation in replicability: A ?many labs\" replication project","author":"Klein R. A.","year":"2014","unstructured":"Klein, R. A., Ratliff, K. A., Vianello, M., Adams Jr., R. B., Bahn\u00edk, S., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., . . . Nosek, B. A. (2014). Data from investigating variation in replicability: A ?many labs\" replication project. Journal of Open Psychology Data, 2(1).","journal-title":"Journal of Open Psychology Data, 2(1)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.17705\/1jais.00302"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1080\/08874417.2020.1779151"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.2307\/2983064"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/PRDC.2015.41"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s42113-019-00029-y"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1037\/0021-9010.86.1.114"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-017--1137-y"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1111\/apps.12124"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2876857"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.17705\/1jais.00526"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082--989X.4.1.84"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1060.0597"},{"key":"e_1_2_1_57_1","first-page":"1","article-title":"Replication research of Moody, Siponen, and Pahnila's unified model of information security policy compliance","volume":"6","author":"Masuch K.","year":"2020","unstructured":"Masuch, K., Hengstler, S., Trang, S., & Brendel, A. B. (2020). Replication research of Moody, Siponen, and Pahnila's unified model of information security policy compliance. AIS Transactions on Replication Research, 6, 1--16.","journal-title":"AIS Transactions on Replication Research"},{"key":"e_1_2_1_58_1","volume-title":"Southern Management Association Meeting, St","author":"Miller B. K.","year":"2008","unstructured":"Miller, B. K., & Chiodo, B. (2008). Academic entitlement: Adapting the equity preference questionnaire for a university setting. Southern Management Association Meeting, St. Pete Beach, Florida."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2013\/37:3.3"},{"key":"e_1_2_1_60_1","volume-title":"Computationally intensive theory construction: A primer for authors and reviewers. MIS Quarterly, 46(2), iii--xviii","author":"Miranda S.","year":"2022","unstructured":"Miranda, S., Berente, N., Seidel, S., Safadi, H., & Burton-Jones, A. (2022). Computationally intensive theory construction: A primer for authors and reviewers. MIS Quarterly, 46(2), iii--xviii."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2018\/13853"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.17705\/1jais.00454"},{"key":"e_1_2_1_63_1","volume-title":"Psychometric theory","author":"Nunnally J. C.","year":"1967","unstructured":"Nunnally, J. C. (1967). Psychometric theory. McGraw-Hill."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.2307\/249774"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1038\/embor.2009.111"},{"key":"e_1_2_1_66_1","volume-title":"SmartPLS 3. SmartPLS GmbH","author":"Ringle C. M.","year":"2015","unstructured":"Ringle, C. M., Wende, S., & Becker, J.-M. (2015). SmartPLS 3. SmartPLS GmbH. B\u00f6nningstedt, Germany."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhm.2019.102447"},{"key":"e_1_2_1_68_1","volume-title":"Diffusion of innovations","author":"Rogers E. M.","year":"2003","unstructured":"Rogers, E. M. (2003). Diffusion of innovations (5th ed.). Free Press.","edition":"5"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1080\/0960085X.2018.1545883"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1214\/10-STS330"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.2307\/23042796"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.5555\/2017470.2017475"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.5555\/2017470.2017476"},{"key":"e_1_2_1_74_1","volume-title":"D. W. Straub, S. Goodman, & R","author":"Straub D. W.","year":"2008","unstructured":"Straub, D. W., & Baskerville, R. (2008). Framing of information security policies and practices. In D. W. Straub, S. Goodman, & R. Baskerville (Eds.), Information security policies, processes and practices. Routledge."},{"key":"e_1_2_1_75_1","volume-title":"Big data: The end of the scientific method? Philosophical Transactions of the Royal Society A, 377(2142)","author":"Succi S.","year":"2019","unstructured":"Succi, S., & Coveney, P. V. (2019). Big data: The end of the scientific method? Philosophical Transactions of the Royal Society A, 377(2142)."},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1177\/1558689806292430"},{"key":"e_1_2_1_77_1","volume-title":"Culture and social behavior","author":"Triandis H. C.","year":"1994","unstructured":"Triandis, H. C. (1994). Culture and social behavior. McGraw-Hill."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2575.2009.00328.x"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/1364636.1364642"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.2307\/41410412"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1080\/00031305.2016.1154108"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2017\/41.3.02"},{"key":"e_1_2_1_83_1","volume-title":"Case study research and applications: Design and methods","author":"Yin R. K.","year":"2018","unstructured":"Yin, R. K. (2018). Case study research and applications: Design and methods. Sage."},{"key":"e_1_2_1_84_1","volume-title":"A conceptual replication of the unified model of information security policy compliance. AIS Transactions on Replication Research, 6","author":"Young D. K.","year":"2020","unstructured":"Young, D. K., Carpenter, D., & McLeod, A. (2020). A conceptual replication of the unified model of information security policy compliance. AIS Transactions on Replication Research, 6."}],"container-title":["ACM SIGMIS Database: the DATABASE for Advances in Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3663682.3663689","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3663682.3663689","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:57:59Z","timestamp":1750294679000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3663682.3663689"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,29]]},"references-count":84,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4,29]]}},"alternative-id":["10.1145\/3663682.3663689"],"URL":"https:\/\/doi.org\/10.1145\/3663682.3663689","relation":{},"ISSN":["0095-0033","1532-0936"],"issn-type":[{"value":"0095-0033","type":"print"},{"value":"1532-0936","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,29]]},"assertion":[{"value":"2024-05-03","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}