{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T17:19:36Z","timestamp":1774027176984,"version":"3.50.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2010,8,23]],"date-time":"2010-08-23T00:00:00Z","timestamp":1282521600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMIS Database"],"published-print":{"date-parts":[[2010,8,23]]},"abstract":"<jats:p>Patterns were originally developed in the field of architecture as a mechanism for communicating good solutions to recurring classes of problems. Since then, many researchers and practitioners have created patterns to describe effective solutions to problems associated with disparate areas such as virtual project management, human-computer interaction, software development and engineering, and design science research. We believe that the development of patterns is a design science activity in which an artifact (i.e., a pattern) is created to communicate about and improve upon the current state-of-practice. Design science research has two critical components, creation and evaluation of an artifact. While many patterns have been created, few, if any, have been evaluated in this sense. In this paper, we propose a framework to evaluate patterns in any domain and provide examples of how to use the evaluation framework. This process of evaluation could help researchers refine extant patterns and improve the possibility of creating sustainable best practices for a given domain. We believe this evaluation framework begins an important dialogue related to the evaluation of patterns as artifacts of design science research. We draw upon the literature associated with patterns, evaluation of design science research, and research methods to develop this framework for evaluating patterns in a more consistent and rigorous manner.<\/jats:p>","DOI":"10.1145\/1851175.1851177","type":"journal-article","created":{"date-parts":[[2010,8,31]],"date-time":"2010-08-31T13:05:55Z","timestamp":1283259955000},"page":"9-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["A design science based evaluation framework for patterns"],"prefix":"10.1145","volume":"41","author":[{"given":"Stacie","family":"Petter","sequence":"first","affiliation":[{"name":"University of Nebraska Omaha, Omaha, NE, Tunisia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deepak","family":"Khazanchi","sequence":"additional","affiliation":[{"name":"University of Nebraska Omaha, Omaha, NE, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John D.","family":"Murphy","sequence":"additional","affiliation":[{"name":"University of Nebraska Omaha, Omaha, NE, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2010,8,23]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Notes on the Synthesis of Form","author":"Alexander C.","unstructured":"Alexander , C. 1964. Notes on the Synthesis of Form , Cambridge : Harvard University Press . Alexander, C. 1964. Notes on the Synthesis of Form, Cambridge: Harvard University Press."},{"key":"e_1_2_1_2_1","volume-title":"The Timeless Way of Building","author":"Alexander C.","unstructured":"Alexander , C. 1979. The Timeless Way of Building , Oxford University Press . Alexander, C. 1979. The Timeless Way of Building, Oxford University Press."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.795104"},{"key":"e_1_2_1_4_1","unstructured":"Alexander C. Ishikawa S. and Silverstein M. 1977. A Pattern Language New York: Oxford University Press.  Alexander C. Ishikawa S. and Silverstein M. 1977. A Pattern Language New York: Oxford University Press."},{"key":"e_1_2_1_5_1","unstructured":"Appleton B. 2000. \"Patterns and Software: Essential Concepts and Terminology. \" Feb. 14 (available online at http:\/\/www.cmcrossroads.com\/bradapp\/docs\/patterns-intro.html)  Appleton B. 2000. \"Patterns and Software: Essential Concepts and Terminology. \" Feb. 14 (available online at http:\/\/www.cmcrossroads.com\/bradapp\/docs\/patterns-intro.html)"},{"key":"e_1_2_1_6_1","unstructured":"Austin R.D. and Westerman G. 2002. \"Destiny WebSolutions Inc. \" Unpbulished Manuscript Boston.  Austin R.D. and Westerman G. 2002. \"Destiny WebSolutions Inc. \" Unpbulished Manuscript Boston."},{"key":"e_1_2_1_7_1","unstructured":"Baskerville R. Pries-Heje J. and Venable J. 2007. \"Soft Design Science Research: Extending the Boundaries of Evaluation in Design Science Research \" 2nd International Conference on Design Science Research in Information Systems & Technology Pasadena CA.  Baskerville R. Pries-Heje J. and Venable J. 2007. \"Soft Design Science Research: Extending the Boundaries of Evaluation in Design Science Research \" 2nd International Conference on Design Science Research in Information Systems & Technology Pasadena CA."},{"key":"e_1_2_1_8_1","unstructured":"Baskerville R. Pries--Heje J. and Venable J. 2008. \"Evaluation Risks in Design Science Research: A Framework \" International Conference on Design Science Research in Information Systems and Technology Atlanta GA.  Baskerville R. Pries--Heje J. and Venable J. 2008. \"Evaluation Risks in Design Science Research: A Framework \" International Conference on Design Science Research in Information Systems and Technology Atlanta GA."},{"key":"e_1_2_1_9_1","first-page":"103","article-title":"Industrial Experience with Design Patterns","author":"Beck K.","year":"1996","unstructured":"Beck , K. , Crocker , R. , Meszaros , G. , Vlissides , J. , Coplien , J.O. , Dominick , L. , and Paulisch , F. 1996 . \" Industrial Experience with Design Patterns ,\" International Conference on Software Engineering, Berlin, Germany , pp. 103 -- 114 . Beck, K., Crocker, R., Meszaros, G., Vlissides, J., Coplien, J.O., Dominick, L., and Paulisch, F. 1996. \"Industrial Experience with Design Patterns,\" International Conference on Software Engineering, Berlin, Germany, pp. 103--114.","journal-title":"International Conference on Software Engineering, Berlin, Germany"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Beck K. and Cunningham W. 1987. \"Using Pattern Languages for Object-Oriented Programs \" OOPSLA Workshop on the Specification and Design for Object-Oriented Programming Orlando FL.  Beck K. and Cunningham W. 1987. \"Using Pattern Languages for Object-Oriented Programs \" OOPSLA Workshop on the Specification and Design for Object-Oriented Programming Orlando FL.","DOI":"10.1145\/28697.28734"},{"key":"e_1_2_1_11_1","unstructured":"Brown W. Malveau R. McCormick H. Mowbray T. and Thomas S.W. 1998. \"The Software Patterns Criteria: Proposed Definitions for Evaluating Software Pattern Quality \" June 10 (available online at http:\/\/www.antipatterns.com\/whatisapattern\/).  Brown W. Malveau R. McCormick H. Mowbray T. and Thomas S.W. 1998. \"The Software Patterns Criteria: Proposed Definitions for Evaluating Software Pattern Quality \" June 10 (available online at http:\/\/www.antipatterns.com\/whatisapattern\/)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Campbell D.T. and Fiske D.W. 1959. \"Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix \" Psychological Bulletin (56:2) pp 81--105.  Campbell D.T. and Fiske D.W. 1959. \"Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix \" Psychological Bulletin (56:2) pp 81--105.","DOI":"10.1037\/h0046016"},{"key":"e_1_2_1_13_1","volume-title":"Chicago, IL: Rand McNally College Publishing Company.","author":"Cook T.D.","year":"1979","unstructured":"Cook , T.D. , and Campbell , D.T . 1979 . Quasi-Experimentation: Design & Analysis Issues for Field Settings , Chicago, IL: Rand McNally College Publishing Company. Cook, T.D., and Campbell, D.T. 1979. Quasi-Experimentation: Design & Analysis Issues for Field Settings, Chicago, IL: Rand McNally College Publishing Company."},{"key":"e_1_2_1_14_1","unstructured":"Coplien J. O. 2007. \"A Pattern Definition: Software Patterns \" (available online at http:\/\/hillside.net\/patterns\/definition.html).  Coplien J. O. 2007. \"A Pattern Definition: Software Patterns \" (available online at http:\/\/hillside.net\/patterns\/definition.html)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/506443.506535"},{"key":"e_1_2_1_16_1","unstructured":"Fernandez A. Holmer T. Rubart J. and Schuemmer T. 2002. \"Three Groupware Patterns from the Activity Awareness Family \" Seventh European Conference on Pattern Languages of Programs EuroPLoP Irsee Germany.  Fernandez A. Holmer T. Rubart J. and Schuemmer T. 2002. \"Three Groupware Patterns from the Activity Awareness Family \" Seventh European Conference on Pattern Languages of Programs EuroPLoP Irsee Germany."},{"key":"e_1_2_1_17_1","volume-title":"Design Patterns: Elements of Reusable Object-Oriented Systems","author":"Gamma E.","year":"1995","unstructured":"Gamma , E. , Helm , R. , Johnson , R. , and Vlissides , J . 1995 . Design Patterns: Elements of Reusable Object-Oriented Systems , Reading, MA : Addison-Wesley . Gamma, E., Helm, R., Johnson, R., and Vlissides, J. 1995. Design Patterns: Elements of Reusable Object-Oriented Systems, Reading, MA: Addison-Wesley."},{"key":"e_1_2_1_18_1","article-title":"The Anatomy of a Design Theory","author":"Gregor S.","year":"2007","unstructured":"Gregor , S. , and Jones , D. 2007 . \" The Anatomy of a Design Theory ,\" Journal of the Association for Information Systems (8:5), pp 312--335. Gregor, S., and Jones, D. 2007. \"The Anatomy of a Design Theory,\" Journal of the Association for Information Systems (8:5), pp 312--335.","journal-title":"Journal of the Association for Information Systems (8:5), pp 312--335."},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Hevner A. March S. Park J. and Ram S. 2004. \"Design Science in Information Systems Research \" MIS Quarterly (28:1) pp 75--105.   Hevner A. March S. Park J. and Ram S. 2004. \"Design Science in Information Systems Research \" MIS Quarterly (28:1) pp 75--105.","DOI":"10.2307\/25148625"},{"key":"e_1_2_1_20_1","volume-title":"Eighth European Conference on Pattern Languages of Programs, EuroPLoP","author":"Homsky O.","year":"2003","unstructured":"Homsky , O. \"More Patterns for Group Leadership,\" in: Eighth European Conference on Pattern Languages of Programs, EuroPLoP , Irsee, Germany , 2003 . Homsky, O. \"More Patterns for Group Leadership,\" in: Eighth European Conference on Pattern Languages of Programs, EuroPLoP, Irsee, Germany, 2003."},{"key":"e_1_2_1_21_1","article-title":"A Paradigmatic Analysis of Information Systems as a Design Science","author":"Iivari J.","year":"2007","unstructured":"Iivari , J. 2007 . \" A Paradigmatic Analysis of Information Systems as a Design Science ,\" Scandinavian Journal of Information Systems (19:2). Iivari, J. 2007. \"A Paradigmatic Analysis of Information Systems as a Design Science,\" Scandinavian Journal of Information Systems (19:2).","journal-title":"Scandinavian Journal of Information Systems (19:2)."},{"key":"e_1_2_1_22_1","first-page":"755","volume-title":"AZ","author":"Khazanchi D.","year":"1996","unstructured":"Khazanchi , D. 1996 . \" A Philosophical Framework for the Validation of Information Systems Concepts,\" Americas Conference on Information Systems, Phoenix , AZ , pp. 755 -- 757 . Khazanchi, D. 1996. \"A Philosophical Framework for the Validation of Information Systems Concepts,\" Americas Conference on Information Systems, Phoenix, AZ, pp. 755--757."},{"key":"e_1_2_1_23_1","volume-title":"PA: Project Management Institute.","author":"Khazanchi D.","year":"2005","unstructured":"Khazanchi , D. , and Zigurs , I . 2005 . Patterns of Effective Management of Virtual Projects: An Exploratory Study, (Newtown Square , PA: Project Management Institute. Khazanchi, D., and Zigurs, I. 2005. Patterns of Effective Management of Virtual Projects: An Exploratory Study, (Newtown Square, PA: Project Management Institute."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.4018\/jec.2006070102"},{"key":"e_1_2_1_25_1","article-title":"A Systematic Method for Discovering Effective Patterns of Virtual Project Management","author":"Khazanchi D.","year":"2007","unstructured":"Khazanchi , D. , and Zigurs , I. 2007 . \" A Systematic Method for Discovering Effective Patterns of Virtual Project Management ,\" Unpublished Working Paper, Social Science Review Network. Khazanchi, D., and Zigurs, I. 2007. \"A Systematic Method for Discovering Effective Patterns of Virtual Project Management,\" Unpublished Working Paper, Social Science Review Network.","journal-title":"Unpublished Working Paper, Social Science Review Network."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.2307\/249410"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/181610.181617"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1700865.1700867"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.11.5.473.15200"},{"key":"e_1_2_1_30_1","unstructured":"Meszaros G. and Doble J. 1996. \"A Pattern Language for Pattern Writing \" Pattern Languages of Programs Conference Allerton Park IL.   Meszaros G. and Doble J. 1996. \"A Pattern Language for Pattern Writing \" Pattern Languages of Programs Conference Allerton Park IL."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1080\/07421222.1990.11517898"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1010061"},{"key":"e_1_2_1_33_1","unstructured":"Pries-Heje J. Baskerville R. and Venable J. 2008. \"Strategies for Design Science Research Evaluation \" European Conference on Information Systems Galway Ireland.  Pries-Heje J. Baskerville R. and Venable J. 2008. \"Strategies for Design Science Research Evaluation \" European Conference on Information Systems Galway Ireland."},{"key":"e_1_2_1_34_1","volume-title":"Real World Research","author":"Robson C.","unstructured":"Robson , C. 2002. Real World Research , ( 2 nd ed.) Malden, MA : Blackwell Publishers . Robson, C. 2002. Real World Research, (2nd ed.) Malden, MA: Blackwell Publishers.","edition":"2"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8116(02)00097-6"},{"key":"e_1_2_1_36_1","unstructured":"Schuemmer T. 2003. \"Evolving a Group Pattern Language \" Fifth International Workshop on Collaborative Editing Helsinki Finland.  Schuemmer T. 2003. \"Evolving a Group Pattern Language \" Fifth International Workshop on Collaborative Editing Helsinki Finland."},{"key":"e_1_2_1_37_1","volume-title":"Handbook of Research Methods: A Guide for Practitioners and Students in the Social Sciences","author":"Sproull N.L.","unstructured":"Sproull , N.L. 1995. Handbook of Research Methods: A Guide for Practitioners and Students in the Social Sciences , ( 2 nd ed.) London : The Scarecrow Press , Inc. Sproull, N.L. 1995. Handbook of Research Methods: A Guide for Practitioners and Students in the Social Sciences, (2nd ed.) London: The Scarecrow Press, Inc.","edition":"2"},{"key":"e_1_2_1_38_1","doi-asserted-by":"crossref","unstructured":"Straub D.W. 1989. \"Validating Instruments in MIS Research \" MIS Quarterly (13:2) pp 146--169.  Straub D.W. 1989. \"Validating Instruments in MIS Research \" MIS Quarterly (13:2) pp 146--169.","DOI":"10.2307\/248922"},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Straub D.W. Boudreau M.-C. and Gefen D. 2004. \"Validation Guidelines for IS Positivist Research \" Communications of the Association for Information Systems (13:Article 24) pp 380--427.  Straub D.W. Boudreau M.-C. and Gefen D. 2004. \"Validation Guidelines for IS Positivist Research \" Communications of the Association for Information Systems (13:Article 24) pp 380--427.","DOI":"10.17705\/1CAIS.01324"},{"key":"e_1_2_1_40_1","volume-title":"Designing Interfaces: Patterns for Effective Interaction Design","author":"Tidwell J.","year":"2005","unstructured":"Tidwell , J. 2005 . Designing Interfaces: Patterns for Effective Interaction Design , Sebastopal, CA : O'Reilly Media . Tidwell, J. 2005. Designing Interfaces: Patterns for Effective Interaction Design, Sebastopal, CA: O'Reilly Media."},{"key":"e_1_2_1_41_1","volume-title":"The Research Methods Knowledge Base, (2 ed.) Cincinnati","author":"Trochim W.","unstructured":"Trochim , W. 2000. The Research Methods Knowledge Base, (2 ed.) Cincinnati , OH : Atomic Dog Publishing . Trochim, W. 2000. The Research Methods Knowledge Base, (2 ed.) Cincinnati, OH: Atomic Dog Publishing."},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Vaishnavi V. and Kuechler W. 2008. \"Design Research in Information Systems \" AISNet (available online at http:\/\/ais.affiniscape.com\/displaycommon.cfm?an=1&subarticlenbr=279).  Vaishnavi V. and Kuechler W. 2008. \"Design Research in Information Systems \" AISNet (available online at http:\/\/ais.affiniscape.com\/displaycommon.cfm?an=1&subarticlenbr=279).","DOI":"10.1504\/JDR.2008.019897"},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Vaishnavi V.K. and Kuechler W. 2007. Improving and Innovating Information & Communication Technology: Design Science Research Methods and Patterns Taylor & Francis.   Vaishnavi V.K. and Kuechler W. 2007. Improving and Innovating Information & Communication Technology: Design Science Research Methods and Patterns Taylor & Francis.","DOI":"10.1201\/9781420059335"},{"key":"e_1_2_1_44_1","first-page":"184","article-title":"A Framework for Design Science Research Activities","author":"Venable J.","year":"2006","unstructured":"Venable , J. 2006 a. \" A Framework for Design Science Research Activities ,\" Information Resource Management Association Conference, Washington D.C. , pp. 184 -- 187 . Venable, J. 2006a. \"A Framework for Design Science Research Activities,\" Information Resource Management Association Conference, Washington D.C., pp. 184--187.","journal-title":"Information Resource Management Association Conference, Washington D.C."},{"key":"e_1_2_1_45_1","unstructured":"Venable J. 2006b. \"The Role of Theory and Theorizing in Design Science Research \" First International Conference on Design Science Research in Information Systems & Technology Claremont CA.  Venable J. 2006b. \"The Role of Theory and Theorizing in Design Science Research \" First International Conference on Design Science Research in Information Systems & Technology Claremont CA."},{"key":"e_1_2_1_46_1","doi-asserted-by":"crossref","unstructured":"Walls J.G. Widmeyer G.R. and El Sawy O.A. 1992. \"Building an Information System Design Theory for Vigilant EIS \" Information Systems Research (3:1) pp 36--59.  Walls J.G. Widmeyer G.R. and El Sawy O.A. 1992. \"Building an Information System Design Theory for Vigilant EIS \" Information Systems Research (3:1) pp 36--59.","DOI":"10.1287\/isre.3.1.36"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1555619.1555635"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.5465\/amr.1989.4308376"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/502795.502798"}],"container-title":["ACM SIGMIS Database: the DATABASE for Advances in Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1851175.1851177","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1851175.1851177","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T12:08:49Z","timestamp":1750248529000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1851175.1851177"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,8,23]]},"references-count":49,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2010,8,23]]}},"alternative-id":["10.1145\/1851175.1851177"],"URL":"https:\/\/doi.org\/10.1145\/1851175.1851177","relation":{},"ISSN":["0095-0033","1532-0936"],"issn-type":[{"value":"0095-0033","type":"print"},{"value":"1532-0936","type":"electronic"}],"subject":[],"published":{"date-parts":[[2010,8,23]]},"assertion":[{"value":"2010-08-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}