{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T16:42:04Z","timestamp":1793378524996,"version":"4.5.0"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T00:00:00Z","timestamp":1790208000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T00:00:00Z","timestamp":1790208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Behav Res"],"DOI":"10.3758\/s13428-026-03184-w","type":"journal-article","created":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T17:57:08Z","timestamp":1790272628000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["An LLM canary in the online data coalmine: Bayesian reasoning problems as a capability-gap test for LLM contamination in online samples"],"prefix":"10.3758","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6470-4109","authenticated-orcid":false,"given":"Vera","family":"Wilde","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,24]]},"reference":[{"issue":"S1","key":"3184_CR1","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1080\/00031305.2018.1543137","volume":"73","author":"V Amrhein","year":"2019","unstructured":"Amrhein, V., Trafimow, D., & Greenland, S. (2019). Inferential statistics as descriptive statistics: There is no replication crisis if we don\u2019t expect replication. The American Statistician, 73(S1), 262\u2013270. https:\/\/doi.org\/10.1080\/00031305.2018.1543137","journal-title":"The American Statistician"},{"issue":"1","key":"3184_CR2","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1016\/j.ijresmar.2021.05.001","volume":"39","author":"AD Arndt","year":"2022","unstructured":"Arndt, A. D., Ford, J. B., Babin, B. J., & Luong, V. (2022). Collecting samples from online services: How to use screeners to improve data quality. International Journal of Research in Marketing, 39(1), 117\u2013133. https:\/\/doi.org\/10.1016\/j.ijresmar.2021.05.001","journal-title":"International Journal of Research in Marketing"},{"issue":"3","key":"3184_CR3","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/0001-6918(80)90046-3","volume":"44","author":"M Bar-Hillel","year":"1980","unstructured":"Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments. Acta Psychologica, 44(3), 211\u2013233. https:\/\/doi.org\/10.1016\/0001-6918(80)90046-3","journal-title":"Acta Psychologica"},{"key":"3184_CR4","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1007\/s10459-020-09986-5","volume":"26","author":"K Binder","year":"2021","unstructured":"Binder, K., Krauss, S., Schmidmaier, R., & Braun, L. T. (2021). Natural frequency trees improve diagnostic efficiency in Bayesian reasoning. Advances in Health Sciences Education, 26, 847\u2013863. https:\/\/doi.org\/10.1007\/s10459-020-09986-5","journal-title":"Advances in Health Sciences Education"},{"issue":"4","key":"3184_CR5","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1177\/1948550619875149","volume":"11","author":"M Chmielewski","year":"2020","unstructured":"Chmielewski, M., & Kucker, S. C. (2020). An MTurk crisis? Shifts in data quality and the impact on study results. Social Psychological and Personality Science, 11(4), 464\u2013473. https:\/\/doi.org\/10.1177\/1948550619875149","journal-title":"Social Psychological and Personality Science"},{"issue":"2","key":"3184_CR6","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1002\/job.1962","volume":"36","author":"JA DeSimone","year":"2015","unstructured":"DeSimone, J. A., Harms, P. D., & DeSimone, A. J. (2015). Best practice recommendations for data screening. Journal of Organizational Behavior, 36(2), 171\u2013181. https:\/\/doi.org\/10.1002\/job.1962","journal-title":"Journal of Organizational Behavior"},{"key":"3184_CR7","doi-asserted-by":"crossref","unstructured":"Dong, Y., Jiang, X., Liu, H., Jin, Z., Gu, B., Yang, M., & Li, G. (2024). Generalization or memorization: Data contamination and trustworthy evaluation for large language models. Findings of the Association for Computational Linguistics: ACL 2024. https:\/\/aclanthology.org\/2024.findings-acl.716\/","DOI":"10.18653\/v1\/2024.findings-acl.716"},{"issue":"2","key":"3184_CR8","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1111\/j.1539-6053.2008.00033.x","volume":"8","author":"G Gigerenzer","year":"2007","unstructured":"Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., & Woloshin, S. (2007). Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest, 8(2), 53\u201396. https:\/\/doi.org\/10.1111\/j.1539-6053.2008.00033.x","journal-title":"Psychological Science in the Public Interest"},{"issue":"4","key":"3184_CR9","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1037\/0033-295X.102.4.684","volume":"102","author":"G Gigerenzer","year":"1995","unstructured":"Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684\u2013704. https:\/\/doi.org\/10.1037\/0033-295X.102.4.684","journal-title":"Psychological Review"},{"key":"3184_CR10","volume-title":"Signal detection theory and psychophysics","author":"DM Green","year":"1966","unstructured":"Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. Wiley."},{"issue":"4","key":"3184_CR11","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/s10654-016-0149-3","volume":"31","author":"S Greenland","year":"2016","unstructured":"Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337\u2013350. https:\/\/doi.org\/10.1007\/s10654-016-0149-3","journal-title":"European Journal of Epidemiology"},{"issue":"2\u20133","key":"3184_CR12","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1017\/S0140525X0999152X","volume":"33","author":"J Henrich","year":"2010","unstructured":"Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2\u20133), 61\u201383. https:\/\/doi.org\/10.1017\/S0140525X0999152X","journal-title":"Behavioral and Brain Sciences"},{"issue":"4","key":"3184_CR13","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1037\/h0034747","volume":"80","author":"D Kahneman","year":"1973","unstructured":"Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237\u2013251. https:\/\/doi.org\/10.1037\/h0034747","journal-title":"Psychological Review"},{"issue":"6","key":"3184_CR14","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1017\/S1930297500006616","volume":"12","author":"M McDowell","year":"2017","unstructured":"McDowell, M., & Jacobs, P. (2017). Meta-analysis of the effect of natural frequencies on Bayesian reasoning. Judgment and Decision Making, 12(6), 548\u2013576. https:\/\/doi.org\/10.1017\/S1930297500006616","journal-title":"Judgment and Decision Making"},{"issue":"3","key":"3184_CR15","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1037\/0096-3445.130.3.380","volume":"130","author":"P Sedlmeier","year":"2001","unstructured":"Sedlmeier, P., & Gigerenzer, G. (2001). Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General, 130(3), 380\u2013400. https:\/\/doi.org\/10.1037\/0096-3445.130.3.380","journal-title":"Journal of Experimental Psychology: General"},{"issue":"11","key":"3184_CR16","doi-asserted-by":"publisher","first-page":"1208","DOI":"10.1038\/s41562-020-0912-z","volume":"4","author":"U Simonsohn","year":"2020","unstructured":"Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour, 4(11), 1208\u20131214. https:\/\/doi.org\/10.1038\/s41562-020-0912-z","journal-title":"Nature Human Behaviour"},{"key":"3184_CR17","first-page":"448","volume-title":"Contributions to probability and statistics: Essays in honor of Harold Hotelling","author":"JW Tukey","year":"1960","unstructured":"Tukey, J. W. (1960). A survey of sampling from contaminated distributions. In I. Olkin, S. G. Ghurye, W. Hoeffding, W. G. Madow, & H. B. Mann (Eds.), Contributions to probability and statistics: Essays in honor of Harold Hotelling (pp. 448\u2013485). Stanford University Press."},{"issue":"3","key":"3184_CR18","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1145\/3685527","volume":"68","author":"V Veselovsky","year":"2025","unstructured":"Veselovsky, V., Horta Ribeiro, M., Cozzolino, P. J., Gordon, A., Rothschild, D., & West, R. (2025). Prevalence and prevention of large language model use in crowd work. Communications of the ACM, 68(3), 42\u201347. https:\/\/doi.org\/10.1145\/3685527","journal-title":"Communications of the ACM"},{"key":"3184_CR19","doi-asserted-by":"publisher","first-page":"1833","DOI":"10.3389\/fpsyg.2018.01833","volume":"9","author":"P Weber","year":"2018","unstructured":"Weber, P., Binder, K., & Krauss, S. (2018). Why can only 24% solve Bayesian reasoning problems in natural frequencies: Frequency phobia as one possible explanation. Frontiers in Psychology, 9, 1833. https:\/\/doi.org\/10.3389\/fpsyg.2018.01833","journal-title":"Frontiers in Psychology"},{"issue":"4","key":"3184_CR20","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1002\/bdm.2182","volume":"34","author":"S Wu","year":"2021","unstructured":"Wu, S., & Emery, N. (2021). American base-rate neglect: It is not the math, but the context. Journal of Behavioral Decision Making, 34(4), 543\u2013564. https:\/\/doi.org\/10.1002\/bdm.2182","journal-title":"Journal of Behavioral Decision Making"}],"container-title":["Behavior Research Methods"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.3758\/s13428-026-03184-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.3758\/s13428-026-03184-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.3758\/s13428-026-03184-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T16:14:50Z","timestamp":1793376890000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/article\/10.3758\/s13428-026-03184-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,24]]},"references-count":20,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2026,11]]}},"alternative-id":["3184"],"URL":"https:\/\/doi.org\/10.3758\/s13428-026-03184-w","relation":{},"ISSN":["1554-3528"],"issn-type":[{"value":"1554-3528","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,24]]},"assertion":[{"value":"10 February 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 September 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 September 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Both studies were conducted on the Prolific platform with informed consent. This research was conducted independently without institutional affiliation; accordingly, no institutional review board reviewed the protocol. Both studies followed Helsinki Declaration principles and Prolific\u2019s participant protection standards.","order":1,"name":"Ethics","label":"Ethics approval","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Informed consent was obtained from all participants via the Prolific platform.","order":2,"name":"Ethics","label":"Consent to participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":3,"name":"Ethics","label":"Consent for publication","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Both pilot studies were preregistered on OSF (\n                      \n                      ). All materials, data, and analysis code are available at\n                      \n                      . Neither study was designed to validate the contamination detection method proposed here; the proposal emerged from post hoc analysis of unexpected results.","order":4,"name":"Ethics","label":"Open practices statement","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author has no relevant financial or non-financial interests to disclose.","order":5,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"301"}}