{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T09:02:01Z","timestamp":1785402121263,"version":"3.56.0"},"reference-count":50,"publisher":"ASME International","issue":"2","license":[{"start":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T00:00:00Z","timestamp":1731369600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Large Language Models (LLMs) have emerged as pivotal technology in the evolving world. Their significance in design lies in their transformative potential to support engineers and collaborate with design teams throughout the design process. However, it is not known whether LLMs can emulate the cognitive and social attributes which are known to be important during design, such as cognitive style. This research evaluates the efficacy of LLMs to emulate aspects of Kirton\u2019s Adaption\u2013Innovation theory, which characterizes individual preferences in problem-solving. Specifically, we use LLMs to generate solutions for three design problems using two different cognitive style prompts (adaptively framed and innovatively framed). Solutions are evaluated with respect to feasibility and paradigm relatedness, which are known to have discriminative value in other studies of cognitive style. We found that solutions generated using the adaptive prompt tend to display higher feasibility and are paradigm-preserving, while solutions generated using the innovative prompts were more paradigm-modifying. This aligns with prior work and expectations for design behavior based on Kirton's Adaption\u2013Innovation theory. Ultimately, these results demonstrate that LLMs can be prompted to accurately emulate cognitive style.<\/jats:p>","DOI":"10.1115\/1.4066857","type":"journal-article","created":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T09:54:21Z","timestamp":1728899661000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":4,"title":["Putting the Ghost in the Machine: Emulating Cognitive Style in Large Language Models"],"prefix":"10.1115","volume":"25","author":[{"given":"Vasvi","family":"Agarwal","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/05x2bcf33","id-type":"ROR","asserted-by":"publisher"}],"name":"Carnegie Mellon University Department of Mechanical Engineering, , 5000, Forbes Avenue, Pittsburgh, PA 15213"},{"name":"Carnegie Mellon University Department of Mechanical Engineering, , 5000, Forbes Avenue, Pittsburgh, PA 15213"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kathryn","family":"Jablokow","sequence":"additional","affiliation":[{"name":"The Pennsylvania State University School of Engineering Design and Innovation, , 30 East, Swedesford Road, Malvern, PA 19355-1488"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"McComb","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University Department of Mechanical Engineering, , 5000 Forbes Avenue, 4126, Wean Hall, Pittsburgh, PA 15213"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"33","published-online":{"date-parts":[[2024,11,12]]},"reference":[{"key":"2024111216135228500_CIT0001","first-page":"V006T06A035","article-title":"Towards Characterizing Cognitive Style Coping Behavior in Engineering Design","author":"Radwan","year":"2022"},{"issue":"2","key":"2024111216135228500_CIT0002","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1111\/j.1467-8691.2010.00553.x","article-title":"How Does Cognitive Conflict in Design Teams Support the Development of Creative Ideas?","volume":"19","author":"Badke-Schaub","year":"2010","journal-title":"Creat. Innov. Manag."},{"key":"2024111216135228500_CIT0003","first-page":"10031","article-title":"Characterization of Project Team Dynamics for Engineering and Management Students Based on Cognitive Style","author":"Buffinton","year":"2002"},{"issue":"6","key":"2024111216135228500_CIT0004","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/0166-4972(91)90019-Z","article-title":"Entrepreneurship and Adaption-Innovation Theory","volume":"11","author":"Goldsmith","year":"1991","journal-title":"Technovation"},{"key":"2024111216135228500_CIT0005","volume-title":"Adaption-Innovation: In the Context of Diversity and Change","author":"Kirton","year":"2003"},{"key":"2024111216135228500_CIT0006","doi-asserted-by":"crossref","DOI":"10.4018\/978-1-4666-6006-9","volume-title":"Approaches to Managing Organizational Diversity and Innovation","author":"Erbe","year":"2014"},{"key":"2024111216135228500_CIT0007","volume-title":"Manual of the Kirton Adaption-Innovation Inventory","author":"Kirton","year":"1977"},{"issue":"1","key":"2024111216135228500_CIT0008","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1002\/j.2162-6057.1996.tb00758.x","article-title":"The Kirton Adaption Innovation Inventory Find Group Problem Solving Success Rates","volume":"30","author":"Hammerschmidt","year":"1996","journal-title":"J. Creat. Behav."},{"key":"2024111216135228500_CIT0009","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2018-85942","article-title":"How Engineering Design Students' Creative Preferences and Cognitive Styles Impact Their Concept Generation and Screening","author":"Heininger","year":"2018"},{"issue":"6","key":"2024111216135228500_CIT0010","doi-asserted-by":"publisher","first-page":"060811","DOI":"10.1115\/1.4062542","article-title":"Deep Learning-Driven Design of Robot Mechanisms","volume":"23","author":"Purwar","year":"2023","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2024111216135228500_CIT0011","first-page":"26.1548.1","article-title":"The Impact of Teaming and Cognitive Style on Student Perceptions of Design Ideation Outcomes","author":"Jablokow","year":"2015"},{"issue":"3","key":"2024111216135228500_CIT0012","doi-asserted-by":"publisher","first-page":"e30","DOI":"10.1002\/brx2.30","article-title":"ChatGPT: The Cognitive Effects on Learning and Memory","volume":"1","author":"Bai","year":"2023","journal-title":"Brain-X"},{"key":"2024111216135228500_CIT0013","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/978-1-4471-5248-4_9","volume-title":"Biologically Inspired Design: Computational Methods and Tools","author":"Linsey","year":"2014"},{"issue":"3","key":"2024111216135228500_CIT0014","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1016\/0166-4972(95)96617-3","article-title":"Kirton Adaption-Innovation Theory: Organizational Implications","volume":"15","author":"Mudd","year":"1995","journal-title":"Technovation"},{"key":"2024111216135228500_CIT0015","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1016\/j.evalprogplan.2016.08.015","article-title":"Concept Mapping \u2013 An Effective Method for Identifying Diversity and Congruity in Cognitive Style","volume":"60","author":"Stoyanov","year":"2017","journal-title":"Eval. Program Plann."},{"issue":"4","key":"2024111216135228500_CIT0016","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1177\/001872678003300401","article-title":"Adaptors and Innovators in Organizations","volume":"33","author":"Kirton","year":"1980","journal-title":"Hum. Relat."},{"issue":"8","key":"2024111216135228500_CIT0017","doi-asserted-by":"publisher","first-page":"1159","DOI":"10.1108\/00251741011076726","article-title":"Adaption-Innovation Theory and Knowledge Use in Organizations","volume":"48","author":"Chilton","year":"2010","journal-title":"Manage. Decis."},{"issue":"5","key":"2024111216135228500_CIT0018","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1037\/0021-9010.61.5.622","article-title":"Adaptors and Innovators: A Description and Measure","volume":"61","author":"Kirton","year":"1976","journal-title":"J. Appl. Psychol."},{"issue":"4","key":"2024111216135228500_CIT0019","doi-asserted-by":"publisher","first-page":"3548","DOI":"10.3758\/s13428-024-02344-0","article-title":"Can Generative AI Infer Thinking Style From Language? Evaluating the Utility of AI as a Psychological Text Analysis Tool","volume":"56","author":"Markowitz","year":"2024","journal-title":"Behav. Res."},{"issue":"4","key":"2024111216135228500_CIT0020","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1177\/1046496404273742","article-title":"Team Role Preference and Cognitive Styles A Convergent Validity Study","volume":"36","author":"Aritzeta","year":"2005","journal-title":"Small Group Res."},{"key":"2024111216135228500_CIT0021","first-page":"603","article-title":"Natural Language Processing","author":"Chowdhary","year":"2020","journal-title":"Fundamentals of Artificial Intelligence"},{"key":"2024111216135228500_CIT0022","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1016\/j.aiopen.2021.12.001","article-title":"A Review of Deep Learning in Question Answering Over Knowledge Bases","volume":"2","author":"Zhang","year":"2021","journal-title":"AI Open"},{"issue":"10","key":"2024111216135228500_CIT0023","doi-asserted-by":"publisher","first-page":"104502","DOI":"10.1115\/1.4054722","article-title":"A Computational Study of Design Team Robustness Through the Lens of Cognitive Style","volume":"144","author":"Radwan","year":"2022","journal-title":"ASME J. Mech. Des."},{"key":"2024111216135228500_CIT0024","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.aiopen.2020.11.001","article-title":"Neural machine translation: A review of methods, resources, and tools","volume":"1","author":"Tan","year":"2020","journal-title":"AI Open"},{"issue":"3","key":"2024111216135228500_CIT0025","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3641289","article-title":"A Survey on Evaluation of Large Language Models","volume":"15","author":"Chang","year":"2024","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"2024111216135228500_CIT0026","author":"Shaki","year":"2023"},{"key":"2024111216135228500_CIT0027","author":"Bosley","year":"2023"},{"issue":"4","key":"2024111216135228500_CIT0028","doi-asserted-by":"publisher","first-page":"041003","DOI":"10.1115\/1.4056220","article-title":"Generative Transformers for Design Concept Generation","volume":"23","author":"Zhu","year":"2023","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2024111216135228500_CIT0029","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2023-116838","article-title":"Conceptual Design Generation Using Large Language Models","author":"Ma","year":"2023"},{"key":"2024111216135228500_CIT0030","doi-asserted-by":"publisher","first-page":"1825","DOI":"10.1017\/pds.2022.185","article-title":"Generative Pre-Trained Transformer for Design Concept Generation: An Exploration","volume":"2","author":"Zhu","year":"2022","journal-title":"Proc. Des. Soc."},{"issue":"6","key":"2024111216135228500_CIT0031","doi-asserted-by":"publisher","first-page":"061004","DOI":"10.1115\/1.4054203","article-title":"Engineering Document Summarization: A Bidirectional Language Model-Based Approach","volume":"22","author":"Qiu","year":"2022","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"5","key":"2024111216135228500_CIT0032","doi-asserted-by":"publisher","DOI":"10.1162\/99608f92.0705d8bd","article-title":"How Can Large Language Models Help Humans in Design And Manufacturing? Part 2: Synthesizing an End-To-End LLM-Enabled Design and Manufacturing Workflow","author":"Makatura","year":"2024","journal-title":"Harvard Data Science Review"},{"key":"2024111216135228500_CIT0033","first-page":"913","article-title":"Supporting Human-AI Collaboration in Auditing LLMs With LLMs","author":"Rastogi","year":"2023"},{"key":"2024111216135228500_CIT0034","doi-asserted-by":"publisher","first-page":"100383","DOI":"10.1016\/j.egyai.2024.100383","article-title":"Opportunities for Large Language Models and Discourse in Engineering Design","volume":"17","author":"G\u00f6pfert","year":"2024","journal-title":"Energy AI"},{"issue":"10","key":"2024111216135228500_CIT0035","doi-asserted-by":"publisher","first-page":"646","DOI":"10.17705\/1jais.00106","article-title":"Identifying Good Ideas: Constructs and Scales for Idea Evaluation","volume":"7","author":"Dean","year":"2006","journal-title":"J. Assoc. Inf. Syst."},{"key":"2024111216135228500_CIT0036","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.destud.2018.08.001","article-title":"Exploring the Effects of Cognitive Style Diversity and Self-Efficacy Beliefs on Final Design Attributes in Student Design Teams","volume":"60","author":"Menold","year":"2019","journal-title":"Des. Stud."},{"key":"2024111216135228500_CIT0037","doi-asserted-by":"publisher","first-page":"1959","DOI":"10.1017\/pds.2024.198","article-title":"Generative Large Language Models in Engineering Design: Opportunities and Challenges","volume":"4","author":"Chiarello","year":"2024","journal-title":"Proc. Des. Soc."},{"key":"2024111216135228500_CIT0038","doi-asserted-by":"publisher","first-page":"e13","DOI":"10.1017\/dsj.2019.12","article-title":"KABOOM: an Agent-Based Model for Simulating Cognitive Style in Team Problem Solving","volume":"5","author":"Lapp","year":"2019","journal-title":"Des. Sci."},{"key":"2024111216135228500_CIT0039","doi-asserted-by":"publisher","DOI":"10.1007\/11402985_15","article-title":"The Traditional Design Process Versus a New Design Methodology","author":"Loudon","year":"2010","journal-title":"IFIP International Federation for Information Processing"},{"issue":"1","key":"2024111216135228500_CIT0040","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.cirp.2024.04.062","article-title":"Systematic Synthesis of Design Prompts for Large Language Models in Conceptual Design","volume":"73","author":"Tian","year":"2024","journal-title":"CIRP Ann."},{"key":"2024111216135228500_CIT0041","author":"Wasi","year":"2024"},{"key":"2024111216135228500_CIT0042","author":"Jadhav","year":"2024"},{"key":"2024111216135228500_CIT0043","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.destud.2019.01.001","article-title":"Crowdsourcing Inspiration: Using Crowd Generated Inspirational Stimuli to Support Designer Ideation","volume":"61","author":"Goucher-Lambert","year":"2019","journal-title":"Des. Stud."},{"key":"2024111216135228500_CIT0044","author":"Brown","year":"2020"},{"key":"2024111216135228500_CIT0045","doi-asserted-by":"crossref","DOI":"10.18260\/p.27156","article-title":"Using Paradigm-Relatedness to Measure Design Ideation Shifts","author":"Silk","year":"2016"},{"key":"2024111216135228500_CIT0046","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2020-22589","article-title":"Best Fits and Dark Horses: Can Design Teams Tell the Difference?","author":"Henderson","year":"2020"},{"issue":"7","key":"2024111216135228500_CIT0047","doi-asserted-by":"publisher","first-page":"072301","DOI":"10.1115\/1.4062393","article-title":"An Exploration of the Relationships Between Cognitive Style, Psychological Safety, and the Paradigm-Relatedness of Design Solutions in Engineering Design Teams in Education","volume":"145","author":"Cole","year":"2023","journal-title":"ASME J. Mech. Des."},{"issue":"1\u20132","key":"2024111216135228500_CIT0048","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1080\/21650349.2018.1463177","article-title":"Incremental to Radical Ideas: Paradigm-Relatedness Metrics for Investigating Ideation Creativity and Diversity","volume":"7","author":"Silk","year":"2019","journal-title":"Int. J. Des. Creat. Innov."},{"issue":"3","key":"2024111216135228500_CIT0049","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1287\/isre.12.3.322.9710","article-title":"Research Report: Modifying Paradigms \u2013 Individual Differences, Creativity Techniques, and Exposure to Ideas in Group Idea Generation","volume":"12","author":"Garfield","year":"2001","journal-title":"Inf. Syst. Res."},{"key":"2024111216135228500_CIT0050","first-page":"1","article-title":"Kruskal-Wallis Test","author":"McKight","year":"2010","journal-title":"The Corsini Encyclopedia of Psychology"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/25\/2\/021002\/7401936\/jcise_25_2_021002.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/25\/2\/021002\/7401936\/jcise_25_2_021002.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T11:14:04Z","timestamp":1731410044000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/25\/2\/021002\/1207208\/Putting-the-Ghost-in-the-Machine-Emulating"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,12]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4066857","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,12]]},"article-number":"021002"}}