{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T06:18:47Z","timestamp":1770877127524,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T00:00:00Z","timestamp":1770076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Behavioral Sciences"],"abstract":"<jats:p>Large language models (LLMs) are increasingly used to predict human behavior from plain-text descriptions of experimental tasks that range from judging disease severity to consequential medical decisions. While these methods promise quick insights without complex psychological theories, we reveal a critical flaw: they often latch onto accidental patterns in the data that seem predictive but collapse when faced with novel experimental conditions. Testing across multiple behavioral studies, we show these models can generate wildly inaccurate predictions, sometimes even reversing true relationships, when applied beyond their training context. Standard validation techniques miss this flaw, creating false confidence in their reliability. We introduce a simple diagnostic tool to spot these failures and urge researchers to prioritize theoretical grounding over statistical convenience. Without this, LLM-driven behavioral predictions risk being scientifically meaningless, despite impressive initial results.<\/jats:p>","DOI":"10.3390\/bs16020219","type":"journal-article","created":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T13:58:49Z","timestamp":1770127129000},"page":"219","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Snake Oil or Panacea? How to Misuse AI in Scientific Inquiries of the Human Mind"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0285-5291","authenticated-orcid":false,"given":"Ren\u00e9","family":"Schlegelmilch","sequence":"first","affiliation":[{"name":"Department of Psychology, Faculty of Human and Health Sciences, University of Bremen, 28359 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7487-1974","authenticated-orcid":false,"given":"Lenard","family":"Dome","sequence":"additional","affiliation":[{"name":"Department of Psychiatry and Psychotherapy, Faculty of Medicine, University T\u00fcbingen, 72074 T\u00fcbingen, Germany"},{"name":"German Center for Mental Health (DZPG), Partner Site T\u00fcbingen, 72074 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1086\/718456","article-title":"Machine learning models for predicting, understanding, and influencing health perception","volume":"7","author":"Aka","year":"2022","journal-title":"Journal of the Association for Consumer Research"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5418","DOI":"10.1038\/s41467-020-18946-z","article-title":"Capturing human categorization of natural images by combining deep networks and cognitive models","volume":"11","author":"Battleday","year":"2020","journal-title":"Nature Communications"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1037\/rev0000047","article-title":"Associative judgment and vector space semantics","volume":"124","author":"Bhatia","year":"2017","journal-title":"Psychological Review"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1037\/rev0000319","article-title":"Transformer networks of human conceptual knowledge","volume":"131","author":"Bhatia","year":"2024","journal-title":"Psychological Review"},{"key":"ref_5","first-page":"e2301239120","article-title":"Predicting implicit attitudes with natural language data","volume":"120","author":"Bhatia","year":"2023","journal-title":"Proceedings of the National Academy of Sciences (PNAS) of the United States of America"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1002","DOI":"10.1038\/s41586-025-09215-4","article-title":"A foundation model to predict and capture human cognition","volume":"644","author":"Binz","year":"2025","journal-title":"Nature"},{"key":"ref_7","unstructured":"Binz, M., and Schulz, E. (2023). Turning large language models into cognitive models. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e385","DOI":"10.1017\/S0140525X22002813","article-title":"Deep problems with neural network models of human vision","volume":"46","author":"Bowers","year":"2023","journal-title":"Behavioral and Brain Sciences"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bowers, J. S., Puebla, G., Thorat, S., Tsetsos, K., and Ludwig, C. (2025). Centaur: A model without a theory., OSF.","DOI":"10.31234\/osf.io\/v9w37_v1"},{"key":"ref_10","unstructured":"Cummins, J. (2025). The threat of analytic flexibility in using large language models to simulate human data: A call to attention. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1037\/rev0000550","article-title":"G-Distance: On the comparison of model and human heterogeneity","volume":"132","author":"Dome","year":"2025","journal-title":"Psychological Review"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Farrell, S., and Lewandowsky, S. (2018). Computational modeling of cognition and behavior, Cambridge University Press.","DOI":"10.1017\/CBO9781316272503"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1038\/s41562-023-01573-1","article-title":"Rethinking model-based and model-free influences on mental effort and striatal prediction errors","volume":"7","author":"Lombardi","year":"2023","journal-title":"Nature Human Behaviour"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"15326","DOI":"10.1038\/s41598-017-15587-z","article-title":"Exploration and recency as the main proximate causes of probability matching: A reinforcement learning analysis","volume":"7","author":"Victorino","year":"2017","journal-title":"Scientific Reports"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"Journal of Statistical Software"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gao, Y., Lee, D., Burtch, G., and Fazelpour, S. (2025). Take caution in using LLMs as human surrogates: Scylla ex machina. arXiv.","DOI":"10.1073\/pnas.2501660122"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/s42113-022-00166-x","article-title":"On logical inference over brains, behaviour, and artificial neural networks","volume":"6","author":"Guest","year":"2023","journal-title":"Computational Brain & Behavior"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Guest, O., and Martin, A. E. (2025). A metatheory of classical and modern connectionism. Psychological Review.","DOI":"10.1037\/rev0000591"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guest, O., and van Rooij, I. (2025). Critical artificial intelligence literacy for psychologists. PsyArXiv, preprint.","DOI":"10.31234\/osf.io\/dkrgj_v1"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction, Springer.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., and Song, D. (2020). Pretrained transformers improve out-of-distribution robustness. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.244"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"8214","DOI":"10.3758\/s13428-024-02455-8","article-title":"A tutorial on open-source large language models for behavioral science","volume":"56","author":"Hussain","year":"2024","journal-title":"Behavior Research Methods"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An introduction to statistical learning: With applications in R, Springer.","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"ref_24","unstructured":"Khromov, G., and Singh, S. P. (2023). Some fundamental aspects about lipschitz continuity of neural networks. arXiv."},{"key":"ref_25","unstructured":"Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2021). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv."},{"key":"ref_26","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A robustly optimized bert pretraining approach. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5319","DOI":"10.1038\/s41598-025-89732-4","article-title":"Coordinating multiple mental faculties during learning","volume":"15","author":"Luo","year":"2025","journal-title":"Scientific Reports"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., and Mian, A. (2024). A comprehensive overview of large language models. arXiv.","DOI":"10.1145\/3744746"},{"key":"ref_29","first-page":"1970","article-title":"Contrasting exemplar and prototype models in a natural-science category domain","volume":"48","author":"Nosofsky","year":"2022","journal-title":"Journal of Experimental Psychology: Learning, Memory, and Cognition"},{"key":"ref_30","unstructured":"Orr, M., Cranford, D., Ford, K., Gluck, K., Hancock, W., Lebiere, C., Pirolli, P., Ritter, F., and Stocco, A. (2025). Not even wrong: On the limits of prediction as explanation in cognitive science. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Peters, M. E., Neumann, M., Zettlemoyer, L., and Yih, W.-T. (2018). Dissecting contextual word embeddings: Architecture and representation. arXiv.","DOI":"10.18653\/v1\/D18-1179"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1037\/0033-295X.113.1.57","article-title":"Global model analysis by parameter space partitioning","volume":"113","author":"Pitt","year":"2006","journal-title":"Psychological Review"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Popper, K. R. (1959). The logic of scientific discovery, Routledge.","DOI":"10.1063\/1.3060577"},{"key":"ref_34","first-page":"109609","article-title":"LLM processes: Numerical predictive distributions conditioned on natural language","volume":"37","author":"Requeima","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1525\/collabra.282","article-title":"Predicting high-level human judgment across diverse domains using word embeddings","volume":"5","author":"Richie","year":"2019","journal-title":"Collabra: Psychology"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1037\/0033-295X.107.2.358","article-title":"How persuasive is a good fit? A comment on theory testing","volume":"107","author":"Roberts","year":"2000","journal-title":"Psychological Review"},{"key":"ref_37","unstructured":"Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2020). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv."},{"key":"ref_38","unstructured":"Schr\u00f6der, S., Morgenroth, T., Kuhl, U., Vaquet, V., and Paa\u00dfen, B. (2025). Large language models do not simulate human psychology. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1002\/bdm.413","article-title":"A re-examination of probability matching and rational choice","volume":"15","author":"Shanks","year":"2002","journal-title":"Journal of Behavioral Decision Making"},{"key":"ref_40","unstructured":"Shuttleworth, R., Andreas, J., Torralba, A., and Sharma, P. (2024). LoRA vs full fine-tuning: An illusion of equivalence. arXiv."},{"key":"ref_41","unstructured":"Singh, P., Peterson, J. C., Battleday, R. M., and Griffiths, T. L. (2020). End-to-end deep prototype and exemplar models for predicting human behavior. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1037\/h0058201","article-title":"The processes involved in the repeated guessing of alternatives","volume":"30","author":"Skinner","year":"1942","journal-title":"Journal of Experimental Psychology"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/j.2517-6161.1974.tb00994.x","article-title":"Cross-validatory choice and assessment of statistical predictions","volume":"36","author":"Stone","year":"1974","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"key":"ref_44","first-page":"1","article-title":"Understanding LLM embeddings for regression","volume":"2025","author":"Tang","year":"2024","journal-title":"Transactions on Machine Learning Research"},{"key":"ref_45","first-page":"2464","article-title":"Modeling artificial category learning from pixels: Revisiting Shepard, Hovland, and Jenkins (1961) with deep neural networks","volume":"43","author":"Tartaglini","year":"2021","journal-title":"Proceedings of the Annual Meeting of the Cognitive Science Society"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1037\/a0025715","article-title":"On the adequacy of current empirical evaluations of formal models of categorization","volume":"138","author":"Wills","year":"2012","journal-title":"Psychological Bulletin"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"e49547","DOI":"10.7554\/eLife.49547","article-title":"Ten simple rules for the computational modeling of behavioral data","volume":"8","author":"Wilson","year":"2019","journal-title":"eLife"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2074","DOI":"10.1037\/a0038199","article-title":"Humans use directed and random exploration to solve the explore\u2014Exploit dilemma","volume":"143","author":"Wilson","year":"2014","journal-title":"Journal of Experimental Psychology General"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Xie, H., and Zhu, J.-Q. (2025). Centaur may have learned a shortcut that explains away psychological tasks, OSF.","DOI":"10.31234\/osf.io\/u7z4t_v1"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1038\/s44387-025-00011-z","article-title":"Large language models for disease diagnosis: A scoping review","volume":"1","author":"Zhou","year":"2025","journal-title":"NPJ Artificial Intelligence"}],"container-title":["Behavioral Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-328X\/16\/2\/219\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T05:27:39Z","timestamp":1770874059000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-328X\/16\/2\/219"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,3]]},"references-count":50,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["bs16020219"],"URL":"https:\/\/doi.org\/10.3390\/bs16020219","relation":{},"ISSN":["2076-328X"],"issn-type":[{"value":"2076-328X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,3]]}}}