{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T14:51:44Z","timestamp":1787064704565,"version":"build-2736575974"},"reference-count":31,"publisher":"National Academy of Sciences","issue":"9","license":[{"start":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T00:00:00Z","timestamp":1708560000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["www.pnas.org"],"crossmark-restriction":true},"short-container-title":["Proc. Natl. Acad. Sci. U.S.A."],"published-print":{"date-parts":[[2024,2,27]]},"abstract":"<jats:p>We administer a Turing test to AI chatbots. We examine how chatbots behave in a suite of classic behavioral games that are designed to elicit characteristics such as trust, fairness, risk-aversion, cooperation, etc., as well as how they respond to a traditional Big-5 psychological survey that measures personality traits. ChatGPT-4 exhibits behavioral and personality traits that are statistically indistinguishable from a random human from tens of thousands of human subjects from more than 50 countries. Chatbots also modify their behavior based on previous experience and contexts \u201cas if\u201d they were learning from the interactions and change their behavior in response to different framings of the same strategic situation. Their behaviors are often distinct from average and modal human behaviors, in which case they tend to behave on the more altruistic and cooperative end of the distribution. We estimate that they act as if they are maximizing an average of their own and partner\u2019s payoffs.<\/jats:p>","DOI":"10.1073\/pnas.2313925121","type":"journal-article","created":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T13:48:56Z","timestamp":1708609736000},"update-policy":"https:\/\/doi.org\/10.1073\/pnas.cm10313","source":"Crossref","is-referenced-by-count":208,"title":["A Turing test of whether AI chatbots are behaviorally similar to humans"],"prefix":"10.1073","volume":"121","author":[{"given":"Qiaozhu","family":"Mei","sequence":"first","affiliation":[{"name":"School of Information, University of Michigan, Ann Arbor, MI 48109"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yutong","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Information, University of Michigan, Ann Arbor, MI 48109"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Walter","family":"Yuan","sequence":"additional","affiliation":[{"name":"MobLab, Pasadena, CA 91107"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9846-4249","authenticated-orcid":false,"given":"Matthew O.","family":"Jackson","sequence":"additional","affiliation":[{"name":"Department of Economics, Stanford University, Stanford, CA 94305"},{"name":"External Faculty, Santa Fe Institute, Santa Fe, NM 87501"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"341","published-online":{"date-parts":[[2024,2,22]]},"reference":[{"key":"e_1_3_4_1_2","doi-asserted-by":"crossref","unstructured":"A. M. Turing Computing machinery and intelligence. MIND: Quart. Rev. Psychol. Philos. 54 433\u2013460 (1950).","DOI":"10.1093\/mind\/LIX.236.433"},{"key":"e_1_3_4_2_2","first-page":"8","volume-title":"Turing Test Success Marks Milestone in Computing History","author":"Warwick K.","year":"2014","unstructured":"K. Warwick, Turing Test Success Marks Milestone in Computing History (University or Reading Press Release, 2014), p. 8."},{"key":"e_1_3_4_3_2","unstructured":"S. Bubeck et al. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv [Preprint] (2023). http:\/\/arxiv.org\/abs\/2303.12712 (Accessed 28 December 2023)."},{"key":"e_1_3_4_4_2","doi-asserted-by":"crossref","unstructured":"K. Girotra L. Meincke C. Terwiesch K. T. Ulrich Ideas are dimes a dozen: large language models for idea generation in innovation. Available at SSRN: http:\/\/dx.doi.org\/10.2139\/ssrn.4526071. Accessed 28 December 2023.","DOI":"10.2139\/ssrn.4526071"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2316205120"},{"key":"e_1_3_4_6_2","unstructured":"T. Eloundou S. Manning P. Mishkin D. Rock GPTs are GPTs: An early look at the labor market impact potential of large language models. arXiv [Preprint] (2023). http:\/\/arxiv.org\/abs\/2303.10130 (Accessed 28 December 2023)."},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsr2214184"},{"key":"e_1_3_4_8_2","doi-asserted-by":"crossref","unstructured":"L. J. Trautman W. G. Voss S. Shackelford How we learned to stop worrying and love AI: Analyzing the rapid evolution of generative pre-trained transformer (GPT) and its impacts on law business and society. Available at SSRN: http:\/\/dx.doi.org\/10.2139\/ssrn.4516154. Accessed 28 December 2023.","DOI":"10.2139\/ssrn.4516154"},{"key":"e_1_3_4_9_2","unstructured":"R. Bommasani et al. The foundation model transparency index. arXiv [Preprint] (2023). http:\/\/arxiv.org\/abs\/2310.12941 (Accessed 28 December 2023)."},{"key":"e_1_3_4_10_2","doi-asserted-by":"crossref","unstructured":"K. Elkins J. Chun Can GPT-3 pass a writer\u2019s Turing Test? J. Cult. Analyt. 5 (2020).","DOI":"10.22148\/001c.17212"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jrp.2008.12.015"},{"key":"e_1_3_4_12_2","doi-asserted-by":"crossref","unstructured":"M. Almlund A. L. Duckworth J. J. Heckman T. Kautz \u201cPersonality psychology and economics\u201d in Handbook of the Economics of Education E. A. Hanushek S. J. Machin L. Woessmann Eds. (Elsevier Amsterdam 2011) pp. 1\u2013181.","DOI":"10.1016\/B978-0-444-53444-6.00001-8"},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-020-0916-8"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2681(82)90011-7"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1006\/game.1994.1021"},{"key":"e_1_3_4_16_2","doi-asserted-by":"publisher","DOI":"10.1006\/game.1995.1027"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11166-013-9170-z"},{"key":"e_1_3_4_18_2","first-page":"891","article-title":"Cooperation in public-goods experiments: Kindness or confusion?","volume":"85","author":"Andreoni J.","year":"1995","unstructured":"J. Andreoni, Cooperation in public-goods experiments: Kindness or confusion? Am. Econ. Rev. 85, 891\u2013904 (1995).","journal-title":"Am. Econ. Rev."},{"key":"e_1_3_4_19_2","volume-title":"Theory of Games and Economic Behavior","author":"Von Neumann J.","year":"1947","unstructured":"J. Von Neumann, O. Morgenstern, Theory of Games and Economic Behavior (Princeton University Press, ed. 2, 1947).","edition":"2"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1177\/002200275800200301"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.3998\/mpub.20269"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.96.19.10933"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.2307\/2295804"},{"key":"e_1_3_4_24_2","doi-asserted-by":"crossref","unstructured":"A. Tversky D. Kahneman \u201cRational choice and the framing of decisions\u201d in Multiple Criteria Decision Making and Risk Analysis Using Microcomputers B. Karpak S. Zionts Eds. (Springer Berlin Heidelberg 1989) pp. 81\u2013126.","DOI":"10.1007\/978-3-642-74919-3_4"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40881-017-0036-z"},{"key":"e_1_3_4_26_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adj5957"},{"key":"e_1_3_4_27_2","unstructured":"P. Butlin et al. Consciousness in artificial intelligence: Insights from the science of consciousness. arXiv [Preprint] (2023). http:\/\/arxiv.org\/abs\/2308.08708 (Accessed 28 December 2023)."},{"key":"e_1_3_4_28_2","unstructured":"N. Shapira et al. Clever hans or neural theory of mind? Stress testing social reasoning in large language models. arXiv [Preprint] (2023). http:\/\/arxiv.org\/abs\/2305.14763 (Accessed 28 December 2023)."},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1138-y"},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.1162\/daed_a_01915"},{"key":"e_1_3_4_31_2","doi-asserted-by":"crossref","unstructured":"Q. Mei Y. Xie W. Yuan M. O. Jackson Data and code for \u201cA Turing test of whether AI chatbots are behaviorally similar to humans.\u201d GitHub. https:\/\/github.com\/yutxie\/ChatGPT-Behavioral. Accessed 28 December 2023.","DOI":"10.2139\/ssrn.4637354"}],"container-title":["Proceedings of the National Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pnas.org\/doi\/pdf\/10.1073\/pnas.2313925121","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T13:49:50Z","timestamp":1708609790000},"score":1,"resource":{"primary":{"URL":"https:\/\/pnas.org\/doi\/10.1073\/pnas.2313925121"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,22]]},"references-count":31,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,2,27]]}},"alternative-id":["10.1073\/pnas.2313925121"],"URL":"https:\/\/doi.org\/10.1073\/pnas.2313925121","relation":{},"ISSN":["0027-8424","1091-6490"],"issn-type":[{"value":"0027-8424","type":"print"},{"value":"1091-6490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,22]]},"assertion":[{"value":"2023-08-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-04","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2313925121"}}