{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T00:15:29Z","timestamp":1775002529513,"version":"3.50.1"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T00:00:00Z","timestamp":1733097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Res. Metr. Anal."],"abstract":"<jats:p>The proliferation of AI-powered bots and sophisticated fraudsters poses a significant threat to the integrity of scientific studies reliant on online surveys across diverse disciplines, including health, social, environmental and political sciences. We found a substantial decline in usable responses from online surveys from 75 to 10% in recent years due to survey fraud. Monetary incentives attract sophisticated fraudsters capable of mimicking genuine open-ended responses and verifying information submitted months prior, showcasing the advanced capabilities of online survey fraud today. This study evaluates the efficacy of 31 fraud indicators and six ensembles using two agriculture surveys in California. To evaluate the performance of each indicator, we use predictive power and recall. Predictive power is a novel variation of precision introduced in this study, and both are simple metrics that allow for non-academic survey practitioners to replicate our methods. The best indicators included a novel email address score, MinFraud Risk Score, consecutive submissions, opting-out of incentives, improbable location.<\/jats:p>","DOI":"10.3389\/frma.2024.1432774","type":"journal-article","created":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T01:23:45Z","timestamp":1733102625000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies"],"prefix":"10.3389","volume":"9","author":[{"given":"Natalia","family":"Pinz\u00f3n","sequence":"first","affiliation":[]},{"given":"Vikram","family":"Koundinya","sequence":"additional","affiliation":[]},{"given":"Ryan E.","family":"Galt","sequence":"additional","affiliation":[]},{"given":"William O'R.","family":"Dowling","sequence":"additional","affiliation":[]},{"given":"Marcela","family":"Baukloh","sequence":"additional","affiliation":[]},{"given":"Namah C.","family":"Taku-Forchu","sequence":"additional","affiliation":[]},{"given":"Tracy","family":"Schohr","sequence":"additional","affiliation":[]},{"given":"Leslie M.","family":"Roche","sequence":"additional","affiliation":[]},{"given":"Samuel","family":"Ikendi","sequence":"additional","affiliation":[]},{"given":"Mark","family":"Cooper","sequence":"additional","affiliation":[]},{"given":"Lauren E.","family":"Parker","sequence":"additional","affiliation":[]},{"given":"Tapan B.","family":"Pathak","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,12,2]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3390\/electronics11040662","article-title":"Enhanced credit card fraud detection model using machine learning","volume":"11","author":"Alfaiz","year":"2022","journal-title":"Electronics"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e12344","DOI":"10.2196\/12344","article-title":"Fraud detection protocol for web-based research among men who have sex with men: development and descriptive evaluation","volume":"5","author":"Ballard","year":"2019","journal-title":"JMIR Public Health Surveill"},{"key":"B3","doi-asserted-by":"publisher","first-page":"601","DOI":"10.1089\/jayao.2020.0018","article-title":"Exposing the risks of social media recruitment in adolescents and young adults with cancer: #beware","volume":"9","author":"Bell","year":"2020","journal-title":"J. Adolesc. Young Adult Oncol"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.20944\/preprints202203.0243.v2","article-title":"The wasps are clever: keeping out and finding bot answers in internet surveys used for health research","author":"Brainard","year":"2022"},{"key":"B5","doi-asserted-by":"publisher","first-page":"901","DOI":"10.1080\/19419899.2021.1936617","article-title":"Bots and nots: safeguarding online survey research with underrepresented and diverse populations","volume":"13","author":"Bybee","year":"2022","journal-title":"Psychol. Sex."},{"key":"B6","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1097\/QAI.0000000000003035","article-title":"Overcoming challenges of online research: measures to ensure enrollment of eligible participants","volume":"91","author":"Campbell","year":"2022","journal-title":"J Acquir. Immune Defic. Syndr."},{"key":"B7","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1177\/1948550617698203","article-title":"Lie for a dime: when most prescreening responses are honest but most study participants are impostors","volume":"8","author":"Chandler","year":"2017","journal-title":"Soc. Psychol. Personal. Sci"},{"key":"B8","doi-asserted-by":"publisher","first-page":"596","DOI":"10.1177\/0013164410366686","article-title":"Random responding as a threat to the validity of effect size estimates in correlational research","volume":"70","author":"Cred\u00e9","year":"2010","journal-title":"Educ. Psychol. Meas"},{"key":"B9","doi-asserted-by":"publisher","first-page":"e7655","DOI":"10.2196\/resprot.7655","article-title":"Addressing participant validity in a small internet health survey (the restore study): protocol and recommendations for survey response validation","volume":"7","author":"Dewitt","year":"2018","journal-title":"JMIR Res. Protoc"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1002\/9781394260645","author":"Dillman","year":"2014","journal-title":"Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method"},{"key":"B11","doi-asserted-by":"publisher","first-page":"2228","DOI":"10.3758\/s13428-018-1103-y","article-title":"Detecting computer-generated random responding in questionnaire-based data: a comparison of seven indices","volume":"51","author":"Dupuis","year":"2019","journal-title":"Behav Res"},{"key":"B12","doi-asserted-by":"publisher","first-page":"100401","DOI":"10.1016\/j.invent.2021.100401","article-title":"Liar! Liar! Identifying eligibility fraud by applicants in digital health research","volume":"25","author":"Glazer","year":"2021","journal-title":"Internet Interv."},{"key":"B13","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1080\/08897077.2019.1691131","article-title":"Out damn bot, out: Recruiting real people into substance use studies on the internet","volume":"41","author":"Godinho","year":"2020","journal-title":"Substance Abuse"},{"key":"B14","doi-asserted-by":"publisher","first-page":"762","DOI":"10.1002\/aepp.13353","article-title":"Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys","volume":"45","author":"Goodrich","year":"2023","journal-title":"Appl. Econ. Perspect. Policy"},{"key":"B15","doi-asserted-by":"publisher","first-page":"2841","DOI":"10.1007\/s11135-021-01252-1","article-title":"Ensuring survey research data integrity in the era of internet bots","volume":"56","author":"Griffin","year":"2022","journal-title":"Qual. Quant"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3477142","article-title":"Gotta CAPTCHA 'Em All: a survey of 20 years of the human-or-computer dilemma","volume":"192","author":"Guerar","year":"2021","journal-title":"ACM Comput. Surv"},{"key":"B17","doi-asserted-by":"publisher","first-page":"e28232","DOI":"10.2196\/28232","article-title":"Methods for authenticating participants in fully web-based mobile app trials from the ireach project: cross-sectional study","volume":"9","author":"Guest","year":"2021","journal-title":"JMIR mHealth uHealth"},{"key":"B18","doi-asserted-by":"publisher","first-page":"e38732","DOI":"10.2196\/38732","article-title":"The vaping and patterns of e-cigarette use research study: protocol for a web-based cohort study","volume":"12","author":"Hardesty","year":"2023","journal-title":"JMIR Res. Protoc"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1395547","DOI":"10.3389\/fsufs.2024.1395547","article-title":"Climate smart agriculture: assessing needs and perceptions of California's farmers","volume":"8","author":"Ikendi","year":"2024","journal-title":"Front. Sustain. Food Syst"},{"key":"B20","unstructured":"Model-agnostic unsupervised detection of bots in a Likert-type questionnaire\n          \n          \n            \n              Ilagan\n              M. J.\n            \n            \n              Falk\n              C. F.\n            \n          \n          37985637\n          PsyArXiv\n          \n          2022"},{"key":"B21","doi-asserted-by":"publisher","first-page":"112289","DOI":"10.1016\/j.paid.2023.112289","article-title":"Bots are the new fraud: a post-hoc exploration of statistical methods to identify bot-generated responses in a corrupt data set","volume":"213","author":"Irish","year":"2023","journal-title":"Pers. Individ. Dif"},{"key":"B22","unstructured":"Preventing Bad Data: How to Catch Survey Speeders, Cheaters & Bots\n          \n          2021"},{"key":"B23","unstructured":"The State of Online Research Panels\n          \n          2023"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1050","DOI":"10.1093\/poq\/nfab057","article-title":"Strategies for detecting insincere respondents in online polling","volume":"85","author":"Kennedy","year":"2021","journal-title":"Public Opin. Q"},{"key":"B25","doi-asserted-by":"publisher","first-page":"614","DOI":"10.1017\/psrm.2020.6","article-title":"The shape of and solutions to the MTurk quality crisis","volume":"8","author":"Kennedy","year":"2020","journal-title":"Polit. Sci. Res. Methods"},{"key":"B26","doi-asserted-by":"publisher","first-page":"205979912110504","DOI":"10.1177\/20597991211050467","article-title":"Suspicious and fraudulent online survey participation: introducing the REAL framework","volume":"14","author":"Lawlor","year":"2021","journal-title":"Methodol. Innov"},{"key":"B27","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1016\/j.pedhc.2022.12.006","article-title":"Methodological challenge: addressing bots in online research","volume":"37","author":"Lawrence","year":"2023","journal-title":"J. Pediatr. Health Care"},{"key":"B28","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1093\/ntr\/ntac194","article-title":"Ongoing refinement and adaptation are required to address participant deception in online nicotine and tobacco research studies","volume":"25","author":"LePine","year":"2023","journal-title":"Nicotine Tob. Res"},{"key":"B29","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1177\/08901171211037531","article-title":"Survey fraud and the integrity of web-based survey research","volume":"36","author":"Levi","year":"2022","journal-title":"Am. J. Health Promot."},{"key":"B30","doi-asserted-by":"publisher","first-page":"e15079","DOI":"10.2196\/15079","article-title":"Recommendations from a descriptive evaluation to improve screening procedures for web-based studies with couples: cross-sectional study","volume":"6","author":"Mitchell","year":"2020","journal-title":"JMIR Public Health Surveill"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1332","DOI":"10.52783\/jes.1508","article-title":"Ai in fraud detection: evaluating the efficacy of artificial intelligence in preventing financial misconduct","volume":"20","author":"Mohan","year":"2024","journal-title":"J. Electr. Syst."},{"key":"B32","unstructured":"Bots or inattentive humans? Identifying sources of low-quality data in online platforms\n          \n          \n            \n              Moss\n              A. J.\n            \n            \n              Rosenzweig\n              C.\n            \n            \n              Jaffe\n              S. N.\n            \n            \n              Gautam\n              R.\n            \n            \n              Robinson\n              J.\n            \n            \n              LItman\n              L\n            \n          \n          PsyArXiv\n          \n          2021"},{"key":"B33","unstructured":"Ofir\n              P.\n            \n          \n          Market Research Fraud: Distributed Survey Farms Exposed\n          \n          2019"},{"key":"B34","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1016\/j.jesp.2009.03.009","article-title":"Instructional manipulation checks: detecting satisficing to increase statistical power","volume":"45","author":"Oppenheimer","year":"2009","journal-title":"J. Exp. Soc. Psychol"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.31219\/osf.io\/pq65s","article-title":"From data to action: an adaptable crosstabs template for participatory survey data analysis","author":"Pinz\u00f3n","year":"","journal-title":"OSF"},{"key":"B36","unstructured":"Farming and ranching through wildfire: producers' critical role in fire risk management and emergency response\n          \n          \n            \n              Pinz\u00f3n\n              N.\n            \n            \n              Galt\n              R.\n            \n            \n              Roche\n              L.\n            \n            \n              Schohr\n              T.\n            \n            \n              Koundinya\n              V.\n            \n            \n              Brimm\n              K.\n            \n          \n          SocArXiv"},{"key":"B37","doi-asserted-by":"publisher","first-page":"e23021","DOI":"10.2196\/23021","article-title":"Threats of bots and other bad actors to data quality following research participant recruitment through social media: cross-sectional questionnaire","volume":"22","author":"Pozzar","year":"2020","journal-title":"J. Med. Internet Res"},{"key":"B38","doi-asserted-by":"publisher","first-page":"e30730","DOI":"10.2196\/30730","article-title":"Strategies for the identification and prevention of survey fraud: data analysis of a web-based survey","volume":"7","author":"Pratt-Chapman","year":"2021","journal-title":"JMIR Cancer"},{"key":"B39","unstructured":"Data Collection Techniques for Quality Outcomes: New Research Uncovers Best Practices for Primary Researchers\n          \n          2021"},{"key":"B40","doi-asserted-by":"publisher","first-page":"164","DOI":"10.3390\/computers10120164","article-title":"Click fraud in digital advertising: a comprehensive survey","volume":"10","author":"Sadeghpour","year":"2021","journal-title":"Computers"},{"key":"B41","unstructured":"Yes stormtrooper, these are the droids you're looking for: a method paper evaluating bot detection strategies in online psychological research\n          \n          \n            \n              Shaw\n              T.\n            \n            \n              Cascalheira\n              C. J.\n            \n          \n          PsyArXiv\n          \n          2023"},{"key":"B42","doi-asserted-by":"publisher","first-page":"472","DOI":"10.20982\/tqmp.16.5.p472","article-title":"Got bots? Practical recommendations to protect online survey data from bot attacks","volume":"16","author":"Storozuk","year":"2020","journal-title":"TQMP"},{"key":"B43","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1111\/jlme.12200","article-title":"Detecting, preventing, and responding to \u201cfraudsters\u201d in internet research: ethics and tradeoffs","volume":"43","author":"Teitcher","year":"2015","journal-title":"J. Law Med. Ethics"},{"key":"B44","doi-asserted-by":"publisher","first-page":"804","DOI":"10.1080\/08941920.2022.2081392","article-title":"An inventory and assessment of sample sources for survey research with agricultural producers in the U.S","volume":"35","author":"Ulrich-Schad","year":"2022","journal-title":"Soc. Nat. Resour"},{"key":"B45","unstructured":"Velo Higueras\n              M.\n            \n          \n          I've been catfished, now what? A case study of identify fraud in online research\n          \n          2023"},{"key":"B46","unstructured":"Accessing Venmo While Abroad\n          \n          2022"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1650","DOI":"10.1111\/cobi.13747","article-title":"Conducting conservation social science surveys online","volume":"35","author":"Wardropper","year":"2021","journal-title":"Conserv. Biol."},{"key":"B48","unstructured":"Zelle FAQ: Can I use Zelle\n          \n          2023"},{"key":"B49","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1145\/3485447.3512230","article-title":"\u201cBeyond bot detection: combating fraudulent online survey takers,\u201d","volume-title":"Proceedings of the ACM Web Conference 2022","author":"Zhang","year":"2022"},{"key":"B50","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1146\/annurev-orgpsych-120920-052946","article-title":"Innovations in sampling: improving the appropriateness and quality of samples in organizational research","volume":"10","author":"Zickar","year":"2022","journal-title":"Annu. Rev. Organ. Psychol. Organ. Behav"}],"container-title":["Frontiers in Research Metrics and Analytics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2024.1432774\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T01:23:58Z","timestamp":1733102638000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2024.1432774\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,2]]},"references-count":50,"alternative-id":["10.3389\/frma.2024.1432774"],"URL":"https:\/\/doi.org\/10.3389\/frma.2024.1432774","relation":{"has-preprint":[{"id-type":"doi","id":"10.31235\/osf.io\/95tka","asserted-by":"object"}]},"ISSN":["2504-0537"],"issn-type":[{"value":"2504-0537","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,2]]},"article-number":"1432774"}}