{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T04:50:20Z","timestamp":1783659020865,"version":"3.55.0"},"reference-count":80,"publisher":"MIT Press","issue":"3","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,2,17]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Large language models (LLMs) have been transformative. They are pretrained foundational models that are self-supervised and can be adapted with fine-tuning to a wide range of natural language tasks, each of which previously would have required a separate network model. This is one step closer to the extraordinary versatility of human language. GPT-3 and, more recently, LaMDA, both of them LLMs, can carry on dialogs with humans on many topics after minimal priming with a few examples. However, there has been a wide range of reactions and debate on whether these LLMs understand what they are saying or exhibit signs of intelligence. This high variance is exhibited in three interviews with LLMs reaching wildly different conclusions. A new possibility was uncovered that could explain this divergence. What appears to be intelligence in LLMs may in fact be a mirror that reflects the intelligence of the interviewer, a remarkable twist that could be considered a reverse Turing test. If so, then by studying interviews, we may be learning more about the intelligence and beliefs of the interviewer than the intelligence of the LLMs. As LLMs become more capable, they may transform the way we interact with machines and how they interact with each other. Increasingly, LLMs are being coupled with sensorimotor devices. LLMs can talk the talk, but can they walk the walk? A road map for achieving artificial general autonomy is outlined with seven major improvements inspired by brain systems and how LLMs could in turn be used to uncover new insights into brain function.<\/jats:p>","DOI":"10.1162\/neco_a_01563","type":"journal-article","created":{"date-parts":[[2023,2,6]],"date-time":"2023-02-06T23:25:54Z","timestamp":1675725954000},"page":"309-342","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":125,"title":["Large Language Models and the Reverse Turing Test"],"prefix":"10.1162","volume":"35","author":[{"given":"Terrence J.","family":"Sejnowski","sequence":"first","affiliation":[{"name":"Salk Institute for Biological Studies, La Jolla, CA 92093, U.S.A."},{"name":"Division of Biological Sciences, University of California, San Diego, La Jolla, CA 92037, U.S.A. terry@salk.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,2,17]]},"reference":[{"key":"2023021720442350200_","volume-title":"Flatland: A romance in many dimensions","author":"Abbott","year":"1884"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1207\/s15516709cog0901_7","article-title":"A learning algorithm for Boltzmann machines","volume":"9","author":"Ackley","year":"1985","journal-title":"Cognitive Science"},{"key":"2023021720442350200_","article-title":"Artificial neural networks are making strides towards consciousness","author":"Ag\u00fcera y Arcas","year":"2022","journal-title":"Economist"},{"key":"2023021720442350200_","article-title":"Can machines learn how to behave?","author":"Ag\u00fcera y Arcas","year":"2022","journal-title":"Medium"},{"key":"2023021720442350200_","article-title":"Evolving brains","author":"Allman","year":"1999","journal-title":"Scientific American Library"},{"key":"2023021720442350200_","article-title":"AI and compute","volume-title":"OpenAI Blog","author":"Amodei","year":"2018"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511613869","volume-title":"The language organ: Linguistics as cognitive physiology","author":"Anderson","year":"2002"},{"key":"2023021720442350200_","first-page":"3","article-title":"The mirror system hypothesis","volume-title":"Action to language via the mirror neuron system","author":"Arbib","year":"2010"},{"key":"2023021720442350200_","first-page":"2285","article-title":"Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks","volume":"20","author":"Bartlett","year":"2019","journal-title":"Journal of Machine Learning Research"},{"key":"2023021720442350200_","article-title":"Dota 2 with large scale deep reinforcement learning","author":"Berner","year":"2019"},{"key":"2023021720442350200_","volume-title":"Why youth is not wasted on the young: Immaturity in human development","author":"Bjorklund","year":"2007"},{"key":"2023021720442350200_","article-title":"The model is the message","volume-title":"Noema Magazine","author":"Bratton","year":"2022"},{"issue":"9","key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"1202","DOI":"10.1016\/S0960-9822(02)70689-1","article-title":"Francisco Crick in Paradiso","volume":"6","author":"Brenner","year":"1996","journal-title":"Current Biology"},{"key":"2023021720442350200_","article-title":"Language models are few-shot learners","volume-title":"Advances in neural information processing systems","author":"Brown","year":"2020"},{"issue":"11","key":"2023021720442350200_","first-page":"18","article-title":"The case against B. F. Skinner","volume":"7","author":"Chomsky","year":"1971","journal-title":"New York Review of Books"},{"key":"2023021720442350200_","volume-title":"Knowledge of language: Its nature, origins, and use","author":"Chomsky","year":"1986"},{"key":"2023021720442350200_","article-title":"PalM: Scaling language modeling with pathways","author":"Chowdhery","year":"2022"},{"key":"2023021720442350200_","volume-title":"Conscience: The origins of moral intuition","author":"Churchland","year":"2019"},{"key":"2023021720442350200_","article-title":"Language models show human-like content effects on reasoning","author":"Dasgupta","year":"2022"},{"issue":"1\u20132","key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/S0010-0277(00)00123-2","article-title":"Towards a cognitive neuroscience of consciousness: Basic evidence and a workspace framework","volume":"79","author":"Dehaene","year":"2001","journal-title":"Cognition"},{"key":"2023021720442350200_","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018"},{"key":"2023021720442350200_","volume-title":"Are we smart enough to know how smart animals are?","author":"de Waal","year":"2016"},{"key":"2023021720442350200_","article-title":"M\u00e9moire sur la propagation de la Chaleur dans les corps solides (Treatise on the propagation of heat in solid bodies)","author":"Fourier","year":"1808"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.1101\/214262","article-title":"A theory of multineuronal dimensionality, dynamics and measurement","author":"Gao","year":"2017"},{"key":"2023021720442350200_","volume-title":"The scientist in the crib: What early learning tells us about the mind","author":"Gopnik","year":"1999"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1093\/schbul\/23.3.459","article-title":"The basal ganglia and cognitive pattern generators","volume":"23","author":"Graybiel","year":"1997","journal-title":"Schizophrenia Bulletin"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/j.neuron.2017.06.011","article-title":"Neuroscience-inspired artificial intelligence","volume":"95","author":"Hassabis","year":"2017","journal-title":"Neuron"},{"key":"2023021720442350200_","first-page":"2908","article-title":"Replay in deep learning: Current approaches and missing biological elements","volume":"33","author":"Hayes","year":"2021","journal-title":"Neural Computation"},{"key":"2023021720442350200_","article-title":"Training compute-optimal large language models","author":"Hoffmann","year":"2022"},{"key":"2023021720442350200_","article-title":"Artificial neural networks are making strides towards consciousness","author":"Hofstadter","year":"2022","journal-title":"Economist"},{"key":"2023021720442350200_","article-title":"Inner monologue: Embodied reasoning through planning with language models","author":"Huang","year":"2022"},{"key":"2023021720442350200_","article-title":"AI personification: Estimating the personality of language models","author":"Karra","year":"2022"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"R1057","DOI":"10.1016\/j.cub.2013.10.051","article-title":"What we know currently about mirror neurons","volume":"2","author":"Kilner","year":"2013","journal-title":"Current Biology"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"issue":"11","key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proceedings of the IEEE"},{"key":"2023021720442350200_","article-title":"Is LaMDA sentient?: An interview","author":"Lemoine","year":"2022","journal-title":"Medium"},{"issue":"7","key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/3490443","article-title":"Language models: Past, present, and future","volume":"65","author":"Li","year":"2022","journal-title":"Communications of the ACM"},{"key":"2023021720442350200_","article-title":"Internal feedback in biological control: Locality and system level synthesis","author":"Li","year":"2022"},{"key":"2023021720442350200_","article-title":"Artificial intelligence: A general survey","volume-title":"Artificial Intelligence: A paper symposium","author":"Lighthill","year":"1973"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1126\/science.1237905","article-title":"Global epigenomic reconfiguration during mammalian brain development","volume":"341","author":"Lister","year":"2013","journal-title":"Science"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.1126\/scirobotics.abo0235","article-title":"From motor control to team play in simulated humanoid football","volume":"7","author":"Liu","year":"2022","journal-title":"Science Robotics"},{"key":"2023021720442350200_","first-page":"44","article-title":"Artificial confidence","author":"Marcus","year":"2022","journal-title":"Scientific American"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1038\/s41586-021-04362-w","article-title":"Brain-inspired computing needs a master plan","volume":"604","author":"Mehonic","year":"2022","journal-title":"Nature"},{"key":"2023021720442350200_","first-page":"246","article-title":"Hierarchical probabilistic neural network language model","volume-title":"Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics","author":"Morin","year":"2005"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"e1916367118","DOI":"10.1073\/pnas.1916367118","article-title":"Diversity-enabled sweet spots in layered architectures and speed-accuracy trade-offs in sensorimotor control","volume":"118","author":"Nakahira","year":"2021","journal-title":"Proceedings of the National Academy of Sciences U.S.A"},{"key":"2023021720442350200_","article-title":"Why animal extinction is crippling computer science","author":"Navlakha","year":"2018","journal-title":"Wired"},{"key":"2023021720442350200_","author":"NETtalk","year":"1986"},{"issue":"1","key":"2023021720442350200_","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.cell.2021.11.037","article-title":"BRAIN 2.0: Transforming neuroscience","volume":"185","author":"Ngai","year":"2022","journal-title":"Cell"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"760","DOI":"10.1126\/science.abm1574","article-title":"Evolutionary loss of complexity in human vocal anatomy as an adaptation for speech","volume":"377","author":"Nishimura","year":"2022","journal-title":"Science"},{"key":"2023021720442350200_","article-title":"Fine-tuning","author":"OpenAI","year":"2022"},{"key":"2023021720442350200_","volume-title":"The book of why: The new science of cause and effect","author":"Pearl","year":"2018"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1038\/s41562-022-01394-8","article-title":"Intuitive physics learning in a deep-learning model inspired by developmental psychology","volume":"6","author":"Piloto","year":"2022","journal-title":"Nature Human Behaviour"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1017\/S0140525X00036839","article-title":"Beyond modularity: Neural evidence for constructivist principles in development","volume":"17","author":"Quartz","year":"1995","journal-title":"Behavioral and Brain Sciences"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","first-page":"2640","DOI":"10.1016\/j.cell.2022.06.047","article-title":"The application of artificial intelligence to biology and neuroscience","volume":"185","author":"Richards","year":"2022","journal-title":"Cell"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"215","DOI":"10.3389\/fnhum.2014.00215","article-title":"Creativity\u2014the unconscious foundations of the incubation period","volume":"8","author":"Ritter","year":"2014","journal-title":"Frontiers in Human Neuroscience"},{"key":"2023021720442350200_","first-page":"145","article-title":"Parallel networks that learn to pronounce English text","volume":"1","author":"Rosenberg","year":"1987","journal-title":"Complex Systems"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.21236\/AD0256582","volume-title":"Principles of neurodynamics: Perceptrons and the theory of brain mechanics","author":"Rosenblatt","year":"1961"},{"key":"2023021720442350200_","volume-title":"Harry Potter and the sorcerer's stone","author":"Rowling","year":"1997"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by backpropagating errors","volume":"323","author":"Rumelhart","year":"1986","journal-title":"Nature"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/11474.001.0001","volume-title":"The deep learning revolution: Artificial intelligence meets human intelligence","author":"Sejnowski","year":"2018"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","first-page":"257","DOI":"10.12987\/9780300235470-034","article-title":"Dopamine made you do it","volume-title":"Think tank: Forty neuroscientists explore the biological roots of human experience","author":"Sejnowski","year":"2019"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"30033","DOI":"10.1073\/pnas.1907373117","article-title":"The unreasonable effectiveness of deep learning in artificial intelligence","volume-title":"Proceedings of the National Academy of Sciences","author":"Sejnowski","year":"2020"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1038\/scientificamerican1012-54","article-title":"The language of the brain","volume":"307","author":"Sejnowski","year":"2012","journal-title":"Scientific American"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.1109\/IJCNN55064.2022.9891914","article-title":"Compute trends across three eras of machine learning","author":"Sevilla","year":"2022"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1177\/1073858410386727","article-title":"Laryngeal motor cortex and control of speech in humans","volume":"17","author":"Simonyan","year":"2011","journal-title":"Neuroscientist"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/j.tics.2017.02.005","article-title":"The cerebellum: Adaptive prediction for movement and cognition","volume":"21","author":"Sokolov","year":"2017","journal-title":"Trends in Cognitive Sciences"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.physbeh.2011.06.004","article-title":"Allostasis: A model of predictive regulation","volume":"106","author":"Sterling","year":"2012","journal-title":"Physiology and Behavior"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","DOI":"10.1109\/TVCG.2022.3209479","article-title":"Interactive and visual prompt engineering for ad-hoc task adaptation with large language models","author":"Strobelt","year":"2022"},{"key":"2023021720442350200_","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Machine Learning"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal difference learning and TD-Gammon","volume":"38","author":"Tesauro","year":"1995","journal-title":"Communications of the ACM"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/0004-3702(89)90017-9","article-title":"A parallel network that learns to play backgammon","volume":"39","author":"Tesauro","year":"1989","journal-title":"Artificial Intelligence Journal"},{"key":"2023021720442350200_","article-title":"LaMDA: Language Models for Dialog Applications","author":"Thoppilan","year":"2022"},{"issue":"9","key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1016\/j.tics.2017.05.012","article-title":"Mind games: Game engines as an architecture for intuitive physics","volume":"21","author":"Ullman","year":"2017","journal-title":"Trends in Cognitive Science"},{"key":"2023021720442350200_","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1146\/annurev-neuro-110920-035434","article-title":"Theory of the multiregional neocortex: Large-scale 729 neural dynamics and distributed cognition","volume":"45","author":"Wang","year":"2022","journal-title":"Annual Review of Neuroscience"},{"key":"2023021720442350200_","article-title":"Chain of thought prompting elicits reasoning in large language models","author":"Wei","year":"2022"},{"key":"2023021720442350200_","article-title":"Philosophers on GPT-3 (updated with replies by GPT-3)","author":"Weinberg","year":"2020","journal-title":"Daily Nous"},{"key":"2023021720442350200_","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1145\/365153.365168","article-title":"ELIZA: A computer program for the study of natural language communication between man and machine","volume":"9","author":"Weizenbaum","year":"1966","journal-title":"Communications of the ACM"},{"key":"2023021720442350200_","article-title":"Farewell, Marvin Minsky (1927\u20132016)","author":"Wolfram","year":"2016"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/3\/309\/2071839\/neco_a_01563.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/3\/309\/2071839\/neco_a_01563.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T05:22:51Z","timestamp":1679635371000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/3\/309\/114731\/Large-Language-Models-and-the-Reverse-Turing-Test"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,17]]},"references-count":80,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2,17]]},"published-print":{"date-parts":[[2023,2,17]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01563","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,3]]},"published":{"date-parts":[[2023,2,17]]}}}