{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,13]],"date-time":"2026-08-13T22:28:53Z","timestamp":1786660133596,"version":"build-2736575974"},"reference-count":20,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Large language models (LLMs) are increasingly used in qualitative research, but their reliability compared to human analysis, especially on large, non-English datasets, is unclear. Previous studies on older models (like GPT-4) show limitations in nuance and token capacity.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study compared the qualitative analysis capabilities of OpenAI\u2019s GPT-5 and Google\u2019s Gemini 2.5 Pro (Gemini) with a human qualitative analysis. The study uses a large dataset of 317 Dutch newspaper articles from January 1, 2020, to December 31, 2023, investigating the sentiment toward nurses during the COVID-19 pandemic.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n The study used a 2-phase methodology. First, a thematic comparison was conducted where the human researchers, GPT-5, and Gemini independently generated inductive coding trees from the entire corpus. Second, a comparative test was performed where all 3 coders applied a predefined codebook to a 10% stratified random subsample. The human baseline was validated through double-coding by 2 independent researchers, achieving an acceptable intercoder reliability (\u03b1=0.703). The AI analysis was iterative, using model-optimized prompts and an article-by-article approach.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Both AI models successfully identified third-order themes (eg, \u201cHealth care heroes\u201d) consistent with the data. In deductive application, however, both models systematically overcoded compared to the human consensus (181 and 183 codes vs 138), resulting in low intercoder reliability against the human baseline (\u03b1=0.487) for GPT-5 and (\u03b1=0.507) for Gemini.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n This study suggests a potential divergence in analytical logic. The observed coding frequencies indicate that LLMs may default to semantic presence (literal frequency), whereas human coders appear to prioritize interpretive significance (contextual weight), leading to systematic overcoding. Consequently, this article argues that LLMs should not be viewed as autonomous researchers but as high-sensitivity filtering instruments requiring human calibration. This study concludes that AI can serve as a valuable assistant for qualitative researchers. Still, it benefits from a rigorous, iterative, and human-in-the-loop approach to manage methodological friction and ensure nuanced, valid analysis.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/98374","type":"journal-article","created":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T17:30:07Z","timestamp":1783704607000},"page":"e98374-e98374","source":"Crossref","is-referenced-by-count":0,"title":["Analyzing Nurse Sentiment in a Dutch Regional Newspaper During the COVID-19 Pandemic: Comparative Content Analysis of GPT-5, Gemini 2.5 Pro, and Human Coding"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-7485-581X","authenticated-orcid":false,"given":"Bram","family":"van den Berkmortel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1791-3626","authenticated-orcid":false,"given":"Daan","family":"Westra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7142-763X","authenticated-orcid":false,"given":"Rachel","family":"Gifford","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8085-0245","authenticated-orcid":false,"given":"Frank Christian","family":"van de Baan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,13]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.2196\/72815","article-title":"AI in qualitative health research appraisal: comparative study","volume":"9","author":"Landerholm","journal-title":"JMIR Form Res"},{"key":"R2","unstructured":"Georgiou GP . Capabilities of GPT-5 across critical domains: is it the next breakthrough? arXiv. Preprint posted online on 2025. [doi: 10.48550\/arXiv.2508.19259]"},{"issue":"2","key":"R3","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.243217","article-title":"Texts are more than notes, they are data: a glimpse into how machines understand text","volume":"316","author":"Kitamura","journal-title":"Radiology"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.2196\/56500","article-title":"Comparing GPT-4 and human researchers in health care data analysis: qualitative description study","volume":"26","author":"Li","journal-title":"J Med Internet Res"},{"issue":"9","key":"R5","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1177\/10497323241244669","article-title":"Prompts, pearls, imperfections: comparing ChatGPT and a human researcher in qualitative data analysis","volume":"35","author":"Wachinger","journal-title":"Qual Health Res"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/70107","article-title":"Application of large language models in data analysis and medical education for assisted reproductive technology: comparative study","volume":"9","author":"Okuyama","journal-title":"JMIR Form Res"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.1177\/16094069231201504","article-title":"Exploring the use of AI in qualitative analysis: a comparative study of guaranteed income data","volume":"22","author":"Hamilton","journal-title":"Int J Qual Methods"},{"key":"R8","doi-asserted-by":"crossref","unstructured":"Rigouts Terryn A de Lhoneux M . Exploratory study on the impact of english bias of generative large language models in dutch and french. Presented at: Joint International Conference on Computational Linguistics, Language Resources and Evaluation. ELRA and ICCL. Turin, Italy. May 2024.[doi: 10.63317\/5ogbe6wmm74i]","DOI":"10.63317\/5ogbe6wmm74i"},{"key":"R9","unstructured":"Hsiao S . Bringing the best of AI to university students in europe, the middle east and africa at no cost. 2025. URL: https:\/\/blog.google\/products-and-platforms\/products\/gemini\/bringing-the-best-ai-to-university-students-in-europe-the-middle-east-and-africa-at-no-cost\/ [Accessed 07-08-2026]"},{"key":"R10","unstructured":"Rijksoverheid. Maart 2020: Maatregelen tegen verspreiding coronavirus, intelligente lockdown. 2020. URL: https:\/\/www.rijksoverheid.nl\/themas\/overheid-en-democratie\/coronavirus-tijdlijn\/maart-2020-maatregelen-tegen-verspreiding-coronavirus [Accessed 22-06-2026]"},{"key":"R11","doi-asserted-by":"crossref","unstructured":"Corbin J Strauss A . Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 3rd ed. SAGE Publications, Inc; 2008. [doi: 10.4135\/9781452230153] ISBN: 9781452230153","DOI":"10.4135\/9781452230153"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","first-page":"169","DOI":"10.18608\/jla.2025.8575","article-title":"Qualitative coding with GPT-4: where it works better","volume":"12","author":"Liu","journal-title":"J Learn Anal"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1177\/16094069251383739","article-title":"AI as a co-researcher in the qualitative research workflow: transforming human-AI collaboration","volume":"24","author":"Costa","journal-title":"Int J Qual Methods"},{"key":"R14","unstructured":"Montes CM Feldt R Martos CM Ouhbi S Premanandan S Graziotin D . Large language models in thematic analysis: prompt engineering, evaluation, and guidelines for qualitative software engineering research. arXiv. Preprint posted online on Oct 21, 2025. [doi: 10.48550\/arXiv.2510.18456]"},{"key":"R15","doi-asserted-by":"crossref","unstructured":"Krippendorff K . Content Analysis: An Introduction to Its Methodology. SAGE Publications, Inc; 2019. [doi: 10.4135\/9781071878781]","DOI":"10.4135\/9781071878781"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1177\/1609406919899220","article-title":"Intercoder reliability in qualitative research: debates and practical guidelines","volume":"19","author":"O\u2019Connor","journal-title":"Int J Qual Methods"},{"issue":"10","key":"R17","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1097\/ACM.0000000000006134","article-title":"Artificial intelligence to support qualitative data analysis: promises, approaches, pitfalls","volume":"100","author":"Cook","journal-title":"Acad Med"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbr.2026.101103","article-title":"Understanding critical thinking in generative artificial intelligence use: development, validation, and correlates of the critical thinking in AI use scale","volume":"22","author":"Lau","journal-title":"Comput Hum Behav Rep"},{"key":"R19","unstructured":"GPT\u20115.1: a smarter, more conversational chatgpt. OpenAI. 2025. URL: https:\/\/openai.com\/index\/gpt-5-1\/ [Accessed 22-06-2026]"},{"key":"R20","unstructured":"Pichai S Kavukcuoglu D . A new era of intelligence with gemini 3. Google. 2025. URL: https:\/\/blog.google\/products-and-platforms\/products\/gemini\/gemini-3\/#note-from-ceo [Accessed 22-06-2026]"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,8,13]],"date-time":"2026-08-13T22:11:11Z","timestamp":1786659071000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e98374"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,13]]},"references-count":20,"URL":"https:\/\/doi.org\/10.2196\/98374","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,13]]},"article-number":"v28i2e98374"}}