{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T04:29:58Z","timestamp":1772771398822,"version":"3.50.1"},"reference-count":111,"publisher":"Cambridge University Press (CUP)","issue":"4","license":[{"start":{"date-parts":[[2018,10,5]],"date-time":"2018-10-05T00:00:00Z","timestamp":1538697600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AIEDAM"],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Design problems are often presented as structured briefs with detailed constraints and requirements, suggesting a fixed definition. However, past studies have identified the importance of exploring design problems for creative design outcomes. Previous protocol studies of designers has shown that problems can \u201cco-evolve\u201d with the development of solutions during the design process. But to date, little evidence has been provided about<jats:italic>how<\/jats:italic>designers systematically explore presented problems to create better solutions. In this study, we conducted a qualitative analysis of 252 design problems collected from publically available sources, including award-winning product designs and open-source design competitions. This database offers an independent sample of presented problems, designers\u2019 alternative problem descriptions, and innovative solutions. We report the results of this large-scale qualitative analysis aimed at characterizing changes to problems during the design process. Inductive coding was used to identify content patterns in \u201cdiscovered\u201d problem descriptions, with qualitative codes reliably scored by two independent coders. A total of 32 distinct patterns of problem exploration were identified across designers and presented problems. Each pattern is described in the form of a generalized strategy to guide designers as they explore problem spaces. The exploration patterns identified in this study are the first empirical evidence of problem exploration in independent design problems. Further, the presence of exploration patterns in discovered problems is associated with the selection of the corresponding solution as a challenge finalist. These empirically identified strategies for problem exploration may be useful for computational tools supporting designers.<\/jats:p>","DOI":"10.1017\/s0890060418000124","type":"journal-article","created":{"date-parts":[[2018,10,5]],"date-time":"2018-10-05T12:25:12Z","timestamp":1538742312000},"page":"415-430","source":"Crossref","is-referenced-by-count":20,"title":["Evidence of problem exploration in creative designs"],"prefix":"10.1017","volume":"32","author":[{"given":"Jaryn A.","family":"Studer","sequence":"first","affiliation":[]},{"given":"Shanna R.","family":"Daly","sequence":"additional","affiliation":[]},{"given":"Seda","family":"McKilligan","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5889-5167","authenticated-orcid":false,"given":"Colleen M.","family":"Seifert","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2018,10,5]]},"reference":[{"key":"S0890060418000124_ref83","volume-title":"Problem Solving Through Creative Analysis","author":"Rickards","year":"1975"},{"key":"S0890060418000124_ref68","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8667.1996.tb00323.x"},{"key":"S0890060418000124_ref73","first-page":"3","volume-title":"Problem Finding, Problem Solving, and Creativity","author":"Mumford","year":"1994"},{"key":"S0890060418000124_ref70","volume-title":"Problem Solving and Creativity in Individuals and Groups","author":"Maier","year":"1970"},{"key":"S0890060418000124_ref71","doi-asserted-by":"publisher","DOI":"10.1016\/S0377-2217(03)00056-0"},{"key":"S0890060418000124_ref67","doi-asserted-by":"publisher","DOI":"10.1115\/1.4024714"},{"key":"S0890060418000124_ref64","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(03)00003-6"},{"key":"S0890060418000124_ref66","first-page":"1397","volume-title":"Handbook of Industrial and Organizational Psychology","author":"MacCrimmon","year":"1976"},{"key":"S0890060418000124_ref92","article-title":"Are you solving the right problem?","volume":"90","author":"Spradlin","year":"2012","journal-title":"Harvard Business Review"},{"key":"S0890060418000124_ref90","doi-asserted-by":"publisher","DOI":"10.1115\/1.3125205"},{"key":"S0890060418000124_ref79","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2011.07.002"},{"key":"S0890060418000124_ref76","doi-asserted-by":"publisher","DOI":"10.1177\/073428299100900104"},{"key":"S0890060418000124_ref98","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.101.1.140"},{"key":"S0890060418000124_ref88","volume-title":"The Sciences of the Artificial","author":"Simon","year":"1969"},{"key":"S0890060418000124_ref103","doi-asserted-by":"crossref","unstructured":"Weaver J , Wood K and Jensen D (2008) Transformation Facilitators: A Quantitative Analysis of Reconfigurable Products and their Characteristics, Proceedings of the ASME 2008 International Design Engineering Conferences & Computers and Information in Engineering Conference, IDETC\/CIE 2008, Brooklyn, New York.","DOI":"10.1115\/DETC2008-49891"},{"key":"S0890060418000124_ref86","doi-asserted-by":"publisher","DOI":"10.1145\/344949.345077"},{"key":"S0890060418000124_ref97","doi-asserted-by":"publisher","DOI":"10.1086\/296365"},{"key":"S0890060418000124_ref77","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-4972(02)00077-9"},{"key":"S0890060418000124_ref89a","first-page":"161","article-title":"Theories of bounded rationality","volume":"1","author":"Simon","year":"1972","journal-title":"Decision and organization"},{"key":"S0890060418000124_ref89","volume-title":"Models of Thought","author":"Simon","year":"1979"},{"key":"S0890060418000124_ref96","unstructured":"Stumpf S and McDonnell J (1999, April). Relating argument to design problem framing. In Proceedings of the 4th Design Thinking Research Symposium (pp. 245-253). Chicago, Il."},{"key":"S0890060418000124_ref93","doi-asserted-by":"publisher","DOI":"10.1037\/a0014970"},{"key":"S0890060418000124_ref63","first-page":"13","article-title":"Qualitative methods used in the assessment of engineering education","volume":"94","author":"Leydens","year":"2004","journal-title":"Journal of Engineering Education"},{"key":"S0890060418000124_ref72","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2013.11.002"},{"key":"S0890060418000124_ref104","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2016.05.001"},{"key":"S0890060418000124_ref69","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-34925-1_1"},{"key":"S0890060418000124_ref100","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(98)00011-8"},{"key":"S0890060418000124_ref30","first-page":"695","article-title":"Innovative solutions through innovated problems","volume":"34","author":"Daly","year":"2018","journal-title":"International Journal of Engineering Education"},{"key":"S0890060418000124_ref91","unstructured":"Singh V , Walter B , Krager J , Putnam N , Koraishy B , Wood K and Jensen D (2007) Design for Transformation: Theory, Method and Application. Proceedings of the IDETC\/CIE 2007, ASME 2007 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, September, Las Vegas, NV."},{"key":"S0890060418000124_ref39","doi-asserted-by":"publisher","DOI":"10.1111\/j.1948-7169.2009.00020.x"},{"key":"S0890060418000124_ref62","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2006.01.001"},{"key":"S0890060418000124_ref18","volume-title":"Making Hard Decisions with Decision Tools: An Introduction to Decision Analysis","author":"Clemen","year":"2014"},{"key":"S0890060418000124_ref38","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(01)00009-6"},{"key":"S0890060418000124_ref87","volume-title":"Organizational Decision Making","author":"Shull","year":"1970"},{"key":"S0890060418000124_ref29","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.2012.tb00048.x"},{"key":"S0890060418000124_ref80","doi-asserted-by":"publisher","DOI":"10.1002\/0470013192.bsa514"},{"key":"S0890060418000124_ref1","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2011.07.004"},{"key":"S0890060418000124_ref81","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.68.2.243"},{"key":"S0890060418000124_ref80a","first-page":"34","article-title":"Critical thinking: The nature of critical and creative thought","volume":"30","author":"Paul","year":"2006","journal-title":"Journal of Developmental Education"},{"key":"S0890060418000124_ref74","volume-title":"Work Systems Design: The IDEALS Concept","author":"Nadler","year":"1967"},{"key":"S0890060418000124_ref82","doi-asserted-by":"publisher","DOI":"10.1504\/JDR.2004.009842"},{"key":"S0890060418000124_ref23","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(97)00010-0"},{"key":"S0890060418000124_ref7","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(98)00031-3"},{"key":"S0890060418000124_ref11","doi-asserted-by":"publisher","DOI":"10.1207\/S15326934CRJ1334_20"},{"key":"S0890060418000124_ref75","volume-title":"Human Problem Solving","author":"Newell","year":"1972"},{"key":"S0890060418000124_ref99","unstructured":"Unbranded Designs (2015). Ret. from http:\/\/www.UnbrandedDesigns.com (Oct. 28, 2015)."},{"key":"S0890060418000124_ref45","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2003.10.010"},{"key":"S0890060418000124_ref48","doi-asserted-by":"publisher","DOI":"10.1111\/j.2044-8279.1983.tb02563.x"},{"key":"S0890060418000124_ref9","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781107415324.004"},{"key":"S0890060418000124_ref94","doi-asserted-by":"publisher","DOI":"10.1080\/10400419.2005.9651486"},{"key":"S0890060418000124_ref19","volume-title":"Making Sense of Qualitative Data: Complementary Research Strategies","author":"Coffey","year":"1996"},{"key":"S0890060418000124_ref78","volume-title":"Creative Behavior Guidebook","author":"Parnes","year":"1967"},{"key":"S0890060418000124_ref35","doi-asserted-by":"publisher","DOI":"10.1002\/j.2162-6057.1982.tb00326.x"},{"key":"S0890060418000124_ref59","volume-title":"Theory and Practice of Social Planning","author":"Kahn","year":"1969"},{"key":"S0890060418000124_ref102","doi-asserted-by":"publisher","DOI":"10.1115\/1.3470028"},{"key":"S0890060418000124_ref17","first-page":"131","volume-title":"Design Theory and Methodology","author":"Christiaans","year":"1992"},{"key":"S0890060418000124_ref46","doi-asserted-by":"publisher","DOI":"10.1002\/j.2162-6057.1975.tb00552.x"},{"key":"S0890060418000124_ref85","volume-title":"The Reflective Practitioner: How Practitioners Think in Action","author":"Sch\u00f6n","year":"1983"},{"key":"S0890060418000124_ref6","volume-title":"Introduction to Design: Fundamentals of Engineering Design","author":"Asimow","year":"1962"},{"key":"S0890060418000124_ref84","unstructured":"Seevinck J and Lenigas T (2013) Rock, paper, scissors: reflective practice for design process in the novice landscape architect. Proceedings of the 2nd International Conference for Design Education Researchers, 2145\u20132159."},{"key":"S0890060418000124_ref104a","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2011.01.003"},{"key":"S0890060418000124_ref65","volume-title":"Innovating for People: Handbook of Human-Centered Design Methods","year":"2012"},{"key":"S0890060418000124_ref34","doi-asserted-by":"publisher","DOI":"10.1037\/10903-000"},{"key":"S0890060418000124_ref15","doi-asserted-by":"publisher","DOI":"10.1207\/s15327809jls0603_1"},{"key":"S0890060418000124_ref41","doi-asserted-by":"crossref","unstructured":"Dzbor M and Zdrahal Z (2002) Modelling design interpretation as interaction of problem framing and problem solving. Proceedings of DETC'02: ASME 2002 Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Montreal, Canada.","DOI":"10.1115\/DETC2002\/DTM-34009"},{"key":"S0890060418000124_ref58","volume-title":"Design Secrets: Products: Fifty Real-Life Projects Uncovered","year":"2001"},{"key":"S0890060418000124_ref95","doi-asserted-by":"crossref","unstructured":"Studer JA , Yilmaz S , Daly SR and Seifert CM (2016) Cognitive heuristics in defining engineering design problems. In 29th International Conference on Design Theory and Methodology, Volume 7. ASME.","DOI":"10.1115\/DETC2016-59942"},{"key":"S0890060418000124_ref61","volume-title":"Content Analysis: An Introduction to its Methodology","author":"Krippendorf","year":"1980"},{"key":"S0890060418000124_ref57","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2008.01.001"},{"key":"S0890060418000124_ref31","doi-asserted-by":"publisher","DOI":"10.1002\/jee.20048"},{"key":"S0890060418000124_ref42","doi-asserted-by":"publisher","DOI":"10.2307\/427437"},{"key":"S0890060418000124_ref13","doi-asserted-by":"publisher","DOI":"10.1016\/0142-694X(79)90004-8"},{"key":"S0890060418000124_ref60","volume-title":"The New Rational Manager","author":"Kepner","year":"1981"},{"key":"S0890060418000124_ref105","doi-asserted-by":"publisher","DOI":"10.1115\/1.4032219"},{"key":"S0890060418000124_ref54","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2006.11.005"},{"key":"S0890060418000124_ref14","first-page":"21","article-title":"Metaphors in design problem solving: implications for creativity","volume":"1","author":"Casakin","year":"2007","journal-title":"International Journal of Design"},{"key":"S0890060418000124_ref101","unstructured":"Vasconcelos LA , Crilly N , Chen CC , Campos F and Kelner J (2016) What's the benefit of problem exploration? Proceedings of the DESIGN 2016 14th International Design Conference (pp. 89-98). Cavtat, Dubrovnik, Croatia: The Design Society."},{"key":"S0890060418000124_ref5","unstructured":"Archer LB (1968) The structure of design processes. Thesis (Ph.D.) Royal College of Art, London, 1968. Shelfmark: Document Supply DRT 484530. Available on Ethos: http:\/\/ethos.bl.uk\/OrderDetails.do?uin=uk.bl.ethos.484530"},{"key":"S0890060418000124_ref47","volume-title":"The Creative Vision: A Longitudinal Study of Problem Finding in Art","author":"Getzels","year":"1976"},{"key":"S0890060418000124_ref56","doi-asserted-by":"publisher","DOI":"10.1080\/03075078912331377496"},{"key":"S0890060418000124_ref52","volume-title":"A Methodology for Systems Engineering","author":"Hall","year":"1962"},{"key":"S0890060418000124_ref21","volume-title":"Doing Qualitative Research","author":"Crabtree","year":"1999"},{"key":"S0890060418000124_ref53","volume-title":"Design Secrets: Products 2: Fifty Real-Life Design Projects Uncovered (V. 2)","author":"Haller","year":"2006"},{"key":"S0890060418000124_ref4","volume-title":"40 Principles: TRIZ Keys to Technical Innovation","author":"Altshuller","year":"1997"},{"key":"S0890060418000124_ref33","volume-title":"Cognitive Heuristic use in Engineering Design Ideation. Proceedings of the American Society for Engineering Education (AC 2010-1032)","author":"Daly","year":"2010"},{"key":"S0890060418000124_ref49","doi-asserted-by":"publisher","DOI":"10.2307\/798843"},{"key":"S0890060418000124_ref12","unstructured":"Bulsuk KG (2011) An introduction to 5-why. Retrieved from http:\/\/blog.bulsuk.com\/2009\/03\/5-why-finding-root-causes. html#axzz1WBoqfIV6. May 1, 2017."},{"key":"S0890060418000124_ref27","first-page":"91","volume-title":"The Foundations of Aesthetics, Art, and Art Education","author":"Csikszentmihalyi","year":"1988"},{"key":"S0890060418000124_ref32","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.2012.tb01121.x"},{"key":"S0890060418000124_ref51a","article-title":"What is the content of \u201cdesign thinking\u201d? Design heuristics as conceptual repertoire","volume":"32","author":"Gray","year":"2016","journal-title":"International Journal of Engineering Education"},{"key":"S0890060418000124_ref8","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(00)00009-0"},{"key":"S0890060418000124_ref50","doi-asserted-by":"publisher","DOI":"10.1016\/j.aei.2004.09.003"},{"key":"S0890060418000124_ref25","doi-asserted-by":"publisher","DOI":"10.1007\/BF01607156"},{"key":"S0890060418000124_ref2","doi-asserted-by":"publisher","DOI":"10.1007\/s00163-002-0023-z"},{"key":"S0890060418000124_ref24","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2004.06.002"},{"key":"S0890060418000124_ref28","first-page":"1","article-title":"Assessing design heuristics for idea generation in an introductory engineering course","volume":"28","author":"Daly","year":"2012","journal-title":"International Journal of Engineering Education"},{"key":"S0890060418000124_ref37","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2011.07.006"},{"key":"S0890060418000124_ref36","unstructured":"Dorst K (2010) The nature of design thinking. DTRS8 Interpreting Design Thinking: Design Thinking Research Symposium Proceedings (DTRS8), 131\u2013139. http:\/\/hdl.handle.net\/10453\/16590"},{"key":"S0890060418000124_ref43","volume-title":"Strategies for Creative Problem Solving","author":"Fogler","year":"2008"},{"key":"S0890060418000124_ref44a","unstructured":"Fu KK , Yang MC and Wood KL (2015) Design principles: The foundation of design. In ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (pp. V007T06A034-V007T06A034). American Society of Mechanical Engineers."},{"key":"S0890060418000124_ref51","first-page":"63","article-title":"The hierarchy of objectives","volume":"42","author":"Granger","year":"1964","journal-title":"Harvard Business Review"},{"key":"S0890060418000124_ref22","volume-title":"Designing and Conducting Mixed Methods Research","author":"Creswell","year":"2007"},{"key":"S0890060418000124_ref20","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"S0890060418000124_ref44","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(99)00018-6"},{"key":"S0890060418000124_ref24a","first-page":"1","volume-title":"Designerly ways of knowing","author":"Cross","year":"2006"},{"key":"S0890060418000124_ref26","doi-asserted-by":"publisher","DOI":"10.1037\/h0031106"},{"key":"S0890060418000124_ref16","volume-title":"Creativity in Design: The Role of Domain Knowledge in Designing","author":"Christiaans","year":"1992"},{"key":"S0890060418000124_ref3","doi-asserted-by":"crossref","DOI":"10.1201\/9781466593442","volume-title":"Creativity as an Exact Science","author":"Altshuller","year":"1984"},{"key":"S0890060418000124_ref40","doi-asserted-by":"publisher","DOI":"10.1037\/h0093599"},{"key":"S0890060418000124_ref55","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.1060.0200"},{"key":"S0890060418000124_ref10","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.2009.tb01005.x"}],"container-title":["Artificial Intelligence for Engineering Design, Analysis and Manufacturing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0890060418000124","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,12]],"date-time":"2020-11-12T03:20:58Z","timestamp":1605151258000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0890060418000124\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,5]]},"references-count":111,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["S0890060418000124"],"URL":"https:\/\/doi.org\/10.1017\/s0890060418000124","relation":{},"ISSN":["0890-0604","1469-1760"],"issn-type":[{"value":"0890-0604","type":"print"},{"value":"1469-1760","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,10,5]]}}}