{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T07:43:07Z","timestamp":1786174987445,"version":"build-2736575974"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T00:00:00Z","timestamp":1786147200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T00:00:00Z","timestamp":1786147200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100011852","name":"Paracelsus Medical University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100011852","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    This mixed-methods study assessed whether reasoning-enabled large language models (LLMs) can classify stances towards COVID-19 vaccination on X (formerly Twitter) and whether model-generated chain-of-thought (CoT) summaries contain reasoning failures relevant to transparent and auditable public health applications. Zero-shot stance classification by o4-mini and Gemini 2.5 Flash (Gemini) was evaluated on 3,060 rehydrated COVID-19 vaccination tweets against human-annotated labels (positive, negative, neutral). We reported accuracy and macro-F1, measured CoT availability, and qualitatively analysed dual-error cases (tweets misclassified by both models) using Mayring\u2019s content analysis guided by the FUTURE-AI framework. At each model\u2019s best-performing setting, both models reached macro-F1 around 0.8, with o4-mini outperforming Gemini (accuracy 0.819 vs. 0.799, McNemar\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.0015; \u0394macro-F1\u2009=\u20090.020, 95% CI 0.008\u20130.032). Under the reasoning-intensive settings, CoT availability differed: Gemini returned a reasoning summary for all tweets, whereas o4-mini did so for 64.7%. Among 1,981 tweets with CoTs from both models, 295 (14.9%) were dual-errors; in 88.8%, both models produced the same wrong label, suggesting shared failure modes. Qualitatively, both models showed the same errors: target confusion (policy vs. vaccine), literal readings of sarcasm, and label\u2013rationale mismatches, recurring across models despite their markedly different CoT lengths. Reasoning LLMs can therefore classify stance accurately, but their readiness for transparent public health applications depends on whether a CoT is available at all and whether it is coherent with the label it accompanies (label\u2013rationale coherence). CoT availability, label\u2013rationale coherence, and safeguards against systematic reasoning failures offer candidate explainability-readiness metrics, alongside accuracy, for trustworthy digital epidemiology.\n                  <\/jats:p>","DOI":"10.1007\/s10916-026-02449-3","type":"journal-article","created":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T06:52:00Z","timestamp":1786171920000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Beyond Accuracy: A Mixed-Methods Audit of Chain-of-Thought Failures in LLM-Based COVID-19 Vaccine Stance Detection"],"prefix":"10.1007","volume":"50","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-1271-817X","authenticated-orcid":false,"given":"Andreas","family":"Praschk","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9249-4658","authenticated-orcid":false,"given":"Valentin","family":"Fischill-Neudeck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1450-4774","authenticated-orcid":false,"given":"Thomas","family":"Caspari","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8526-2832","authenticated-orcid":false,"given":"Hans-Peter","family":"Wiesinger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,8,8]]},"reference":[{"issue":"3","key":"2449_CR1","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1038\/s41562-021-01056-1","volume":"5","author":"S Loomba","year":"2021","unstructured":"Loomba S, De Figueiredo A, Piatek SJ, De Graaf K, Larson HJ (2021) Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nat Hum Behav 5(3):337\u2013348. https:\/\/doi.org\/10.1038\/s41562-021-01056-1","journal-title":"Nat Hum Behav"},{"key":"2449_CR2","doi-asserted-by":"publisher","first-page":"14778","DOI":"10.1109\/ACCESS.2023.3242234","volume":"11","author":"N Braig","year":"2023","unstructured":"Braig N, Benz A, Voth S, Breitenbach J, Buettner R (2023) Machine Learning Techniques for Sentiment Analysis of COVID-19-Related Twitter Data. IEEE Access 11:14778\u201314803. https:\/\/doi.org\/10.1109\/ACCESS.2023.3242234","journal-title":"IEEE Access"},{"issue":"10","key":"2449_CR3","doi-asserted-by":"publisher","first-page":"1378","DOI":"10.2105\/AJPH.2018.304567","volume":"108","author":"DA Broniatowski","year":"2018","unstructured":"Broniatowski DA, Jamison AM, Qi S et al (2018) Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate. Am J Public Health 108(10):1378\u20131384. https:\/\/doi.org\/10.2105\/AJPH.2018.304567","journal-title":"Am J Public Health"},{"issue":"7","key":"2449_CR4","doi-asserted-by":"publisher","first-page":"e1002616","DOI":"10.1371\/journal.pcbi.1002616","volume":"8","author":"M Salath\u00e9","year":"2012","unstructured":"Salath\u00e9 M, Bengtsson L, Bodnar TJ et al (2012) Digital Epidemiology. PLoS Comput Biol 8(7):e1002616. https:\/\/doi.org\/10.1371\/journal.pcbi.1002616","journal-title":"PLoS Comput Biol"},{"key":"2449_CR5","doi-asserted-by":"publisher","unstructured":"Vaswani A, Shazeer N, Parmar N et al (2017) Attention is All you Need. arXiv. https:\/\/doi.org\/10.48550\/arXiv.1706.03762","DOI":"10.48550\/arXiv.1706.03762"},{"key":"2449_CR6","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv. https:\/\/doi.org\/10.48550\/arXiv.1810.04805","DOI":"10.48550\/arXiv.1810.04805"},{"key":"2449_CR7","doi-asserted-by":"publisher","first-page":"948880","DOI":"10.3389\/fpubh.2022.948880","volume":"10","author":"S Cheatham","year":"2022","unstructured":"Cheatham S, Kummervold PE, Parisi L et al (2022) Understanding the vaccine stance of Italian tweets and addressing language changes through the COVID-19 pandemic: Development and validation of a machine learning model. Front Public Health 10:948880. https:\/\/doi.org\/10.3389\/fpubh.2022.948880","journal-title":"Front Public Health"},{"issue":"11","key":"2449_CR8","doi-asserted-by":"publisher","first-page":"e745","DOI":"10.1016\/S2589-7500(21)00208-9","volume":"3","author":"M Ghassemi","year":"2021","unstructured":"Ghassemi M, Oakden-Rayner L, Beam AL (2021) The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health 3(11):e745\u2013e750. https:\/\/doi.org\/10.1016\/S2589-7500(21)00208-9","journal-title":"Lancet Digit Health"},{"key":"2449_CR9","doi-asserted-by":"publisher","first-page":"102158","DOI":"10.1016\/j.artmed.2021.102158","volume":"124","author":"TP Quinn","year":"2022","unstructured":"Quinn TP, Jacobs S, Senadeera M, Le V, Coghlan S (2022) The three ghosts of medical AI: Can the black-box present deliver? Artif Intell Med 124:102158. https:\/\/doi.org\/10.1016\/j.artmed.2021.102158","journal-title":"Artif Intell Med"},{"key":"2449_CR10","doi-asserted-by":"publisher","unstructured":"Mohammad SM, Kiritchenko S, Sobhani P, Zhu X, Cherry C (2016) SemEval-2016 Task 6: Detecting Stance in Tweets. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). Association for Computational Linguistics :31\u201341. https:\/\/doi.org\/10.18653\/v1\/s16-1003","DOI":"10.18653\/v1\/s16-1003"},{"issue":"4","key":"2449_CR11","doi-asserted-by":"publisher","first-page":"102597","DOI":"10.1016\/j.ipm.2021.102597","volume":"58","author":"A ALDayel","year":"2021","unstructured":"ALDayel A, Magdy W (2021) Stance detection on social media: State of the art and trends. Inf Process Manag 58(4):102597. https:\/\/doi.org\/10.1016\/j.ipm.2021.102597","journal-title":"Inf Process Manag"},{"key":"2449_CR12","doi-asserted-by":"publisher","unstructured":"Lan X, Gao C, Jin D, Li Y (2024) Stance Detection with Collaborative Role-Infused LLM-Based Agents. In: Proceedings of the International AAAI Conference on Web and Social Media, vol 18. AAAI. https:\/\/doi.org\/10.1609\/icwsm.v18i1.31360","DOI":"10.1609\/icwsm.v18i1.31360"},{"key":"2449_CR13","doi-asserted-by":"publisher","unstructured":"He L, Omranian S, McRoy S, Zheng K (2024) Using Large Language Models for sentiment analysis of health-related social media data: empirical evaluation and practical tips. medRxiv. https:\/\/doi.org\/10.1101\/2024.03.19.24304544","DOI":"10.1101\/2024.03.19.24304544"},{"issue":"10","key":"2449_CR14","doi-asserted-by":"publisher","first-page":"2181","DOI":"10.1093\/jamia\/ocae210","volume":"31","author":"Y Guo","year":"2024","unstructured":"Guo Y, Ovadje A, Al-Garadi MA, Sarker A (2024) Evaluating large language models for health-related text classification tasks with public social media data. J Am Med Inform Assoc 31(10):2181\u20132189. https:\/\/doi.org\/10.1093\/jamia\/ocae210","journal-title":"J Am Med Inform Assoc"},{"issue":"10","key":"2449_CR15","doi-asserted-by":"publisher","first-page":"e0000631","DOI":"10.1371\/journal.pdig.0000631","volume":"3","author":"L Espinosa","year":"2024","unstructured":"Espinosa L, Salath\u00e9 M (2024) Use of large language models as a scalable approach to understanding public health discourse. PLOS Digit Health 3(10):e0000631. https:\/\/doi.org\/10.1371\/journal.pdig.0000631","journal-title":"PLOS Digit Health"},{"key":"2449_CR16","doi-asserted-by":"publisher","unstructured":"Brown TB, Mann B, Ryder N et al (2020) Language Models are Few-Shot Learners. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2005.14165","DOI":"10.48550\/arXiv.2005.14165"},{"key":"2449_CR17","doi-asserted-by":"publisher","unstructured":"Wei J, Wang X, Schuurmans D et al (2022) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2201.11903","DOI":"10.48550\/arXiv.2201.11903"},{"key":"2449_CR18","unstructured":"OpenAI (2025) OpenAI o3 and o4-mini System Card. https:\/\/cdn.openai.com\/pdf\/2221c875-02dc-4789-800b-e7758f3722c1\/o3-and-o4-mini-system-card.pdf Accessed 16 Oct 2025"},{"key":"2449_CR19","doi-asserted-by":"publisher","unstructured":"Google DeepMind (2025) Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2507.06261","DOI":"10.48550\/arXiv.2507.06261"},{"key":"2449_CR20","doi-asserted-by":"publisher","unstructured":"Turpin M, Michael J, Perez E, Bowman S (2023) Language Models Don\u2019t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. In: Advances in Neural Information Processing Systems 36. NeurIPS. https:\/\/doi.org\/10.48550\/arXiv.2305.04388","DOI":"10.48550\/arXiv.2305.04388"},{"key":"2449_CR21","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \u201cWhy Should I Trust You?\u201d: Explaining the Predictions of Any Classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM :1135\u20131144. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"2449_CR22","doi-asserted-by":"publisher","unstructured":"Lundberg S, Lee SI (2017) A Unified Approach to Interpreting Model Predictions. arXiv. https:\/\/doi.org\/10.48550\/arXiv.1705.07874","DOI":"10.48550\/arXiv.1705.07874"},{"issue":"2","key":"2449_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3639372","volume":"15","author":"H Zhao","year":"2024","unstructured":"Zhao H, Chen H, Yang F et al (2024) Explainability for Large Language Models: A Survey. ACM Trans Intell Syst Technol 15(2):1\u201338. https:\/\/doi.org\/10.1145\/3639372","journal-title":"ACM Trans Intell Syst Technol"},{"issue":"7","key":"2449_CR24","doi-asserted-by":"publisher","first-page":"1119","DOI":"10.3390\/math12071119","volume":"12","author":"D Ding","year":"2024","unstructured":"Ding D, Dai G, Peng C, Peng X, Zhang B, Huang H (2024) Distantly Supervised Explainable Stance Detection via Chain-of-Thought Supervision. Mathematics 12(7):1119. https:\/\/doi.org\/10.3390\/math12071119","journal-title":"Mathematics"},{"key":"2449_CR25","doi-asserted-by":"publisher","unstructured":"Gatto J, Sharif O, Preum SM (2023) Chain-of-Thought Embeddings for Stance Detection on Social Media. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2310.19750","DOI":"10.48550\/arXiv.2310.19750"},{"issue":"3","key":"2449_CR26","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3390\/informatics11030057","volume":"11","author":"ZA Nazi","year":"2024","unstructured":"Nazi ZA, Peng W (2024) Large Language Models in Healthcare and Medical Domain: A Review. Informatics 11(3):57. https:\/\/doi.org\/10.3390\/informatics11030057","journal-title":"Informatics"},{"key":"2449_CR27","doi-asserted-by":"publisher","first-page":"e081554","DOI":"10.1136\/bmj-2024-081554","volume":"388","author":"K Lekadir","year":"2025","unstructured":"Lekadir K, Frangi AF, Porras AR et al (2025) FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ 388:e081554. https:\/\/doi.org\/10.1136\/bmj-2024-081554","journal-title":"BMJ"},{"key":"2449_CR28","volume-title":"Qualitative Content Analysis: A Step-by-Step Guide","author":"P Mayring","year":"2021","unstructured":"Mayring P (2021) Qualitative Content Analysis: A Step-by-Step Guide. SAGE Publications, London"},{"key":"2449_CR29","volume-title":"Designing and Conducting Mixed Methods Research","author":"JW Creswell","year":"2018","unstructured":"Creswell JW, Plano Clark VL (2018) Designing and Conducting Mixed Methods Research, 3rd edn. SAGE Publications, Thousand Oaks","edition":"3"},{"issue":"19","key":"2449_CR30","doi-asserted-by":"publisher","first-page":"10438","DOI":"10.3390\/ijerph181910438","volume":"18","author":"LA Cotfas","year":"2021","unstructured":"Cotfas LA, Delcea C, Gherai R (2021) COVID-19 Vaccine Hesitancy in the Month Following the Start of the Vaccination Process. Int J Environ Res Public Health 18(19):10438. https:\/\/doi.org\/10.3390\/ijerph181910438","journal-title":"Int J Environ Res Public Health"},{"issue":"1","key":"2449_CR31","doi-asserted-by":"publisher","first-page":"e28749","DOI":"10.2196\/28749","volume":"24","author":"Z Shakeri Hossein Abad","year":"2022","unstructured":"Shakeri Hossein Abad Z, Butler GP, Thompson W, Lee J (2022) Crowdsourcing for Machine Learning in Public Health Surveillance: Lessons Learned From Amazon Mechanical Turk. J Med Internet Res 24(1):e28749. https:\/\/doi.org\/10.2196\/28749","journal-title":"J Med Internet Res"},{"key":"2449_CR32","doi-asserted-by":"publisher","unstructured":"Doshi-Velez F, Kim B (2017) Towards A Rigorous Science of Interpretable Machine Learning. arXiv. https:\/\/doi.org\/10.48550\/arXiv.1702.08608","DOI":"10.48550\/arXiv.1702.08608"},{"issue":"4","key":"2449_CR33","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1007\/s11023-018-9482-5","volume":"28","author":"L Floridi","year":"2018","unstructured":"Floridi L, Cowls J, Beltrametti M et al (2018) AI4People\u2014An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds Mach 28(4):689\u2013707. https:\/\/doi.org\/10.1007\/s11023-018-9482-5","journal-title":"Minds Mach"},{"issue":"1","key":"2449_CR34","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/s41746-020-0253-3","volume":"3","author":"MP Sendak","year":"2020","unstructured":"Sendak MP, Gao M, Brajer N, Balu S (2020) Presenting machine learning model information to clinical end users with model facts labels. npj Digit Med 3(1):41. https:\/\/doi.org\/10.1038\/s41746-020-0253-3","journal-title":"npj Digit Med"},{"issue":"6","key":"2449_CR35","doi-asserted-by":"publisher","first-page":"836","DOI":"10.1097\/EDE.0000000000000919","volume":"29","author":"ADP Chiavegatto Filho","year":"2018","unstructured":"Chiavegatto Filho ADP, Dos Santos HG, Do Nascimento CF, Massa K, Kawachi I (2018) Overachieving Municipalities in Public Health: A Machine-learning Approach. Epidemiology 29(6):836\u2013840. https:\/\/doi.org\/10.1097\/EDE.0000000000000919","journal-title":"Epidemiology"},{"key":"2449_CR36","unstructured":"Pava JN, Meinhardt C, Zaman HBU et al (2025) Mind the (Language) Gap: Mapping the Challenges of LLM Development in Low-Resource Language Contexts. The Asia Foundation, Stanford Institute for Human-Centered Artificial Intelligence, University of Pretoria. https:\/\/hai.stanford.edu\/assets\/files\/hai-taf-pretoria-white-paper-mind-the-language-gap.pdf Accessed 11 Nov 2025"},{"key":"2449_CR37","unstructured":"Franzke AS, Bechmann A, Zimmer M, Ess C, Association of Internet Researchers (2020) Internet Research: Ethical Guidelines 3.0. https:\/\/aoir.org\/reports\/ethics3.pdf Accessed 16 Oct 2025"},{"key":"2449_CR38","doi-asserted-by":"publisher","first-page":"102723","DOI":"10.1016\/j.pmedr.2024.102723","volume":"42","author":"S Kim","year":"2024","unstructured":"Kim S, Kim K, Jo CW (2024) Accuracy of a large language model in distinguishing anti- and pro-vaccination messages on social media: The case of human papillomavirus vaccination. Prev Med Rep 42:102723. https:\/\/doi.org\/10.1016\/j.pmedr.2024.102723","journal-title":"Prev Med Rep"},{"issue":"2","key":"2449_CR39","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1162\/coli_a_00511","volume":"50","author":"Q Lyu","year":"2024","unstructured":"Lyu Q, Apidianaki M, Callison-Burch C (2024) Towards Faithful Model Explanation in NLP: A Survey. Computational Linguistics 50(2):657\u2013723. https:\/\/doi.org\/10.1162\/coli_a_00511","journal-title":"Computational Linguistics"},{"key":"2449_CR40","doi-asserted-by":"publisher","unstructured":"Jacovi A, Goldberg Y (2020) Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics :4198\u20134205. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.386","DOI":"10.18653\/v1\/2020.acl-main.386"},{"key":"2449_CR41","doi-asserted-by":"publisher","unstructured":"Wiegreffe S, Marasovi\u0107 A, Smith NA (2021) Measuring Association Between Labels and Free-Text Rationales. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics :10266\u201310284. https:\/\/doi.org\/10.18653\/v1\/2021.emnlp-main.804","DOI":"10.18653\/v1\/2021.emnlp-main.804"},{"key":"2449_CR42","doi-asserted-by":"publisher","unstructured":"Atanasova P, Camburu O-M, Lioma C, Lukasiewicz T, Simonsen JG, Augenstein I (2023) Faithfulness Tests for Natural Language Explanations. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics :283\u2013294. https:\/\/doi.org\/10.18653\/v1\/2023.acl-short.25","DOI":"10.18653\/v1\/2023.acl-short.25"},{"key":"2449_CR43","doi-asserted-by":"publisher","unstructured":"Paul D, West R, Bosselut A, Faltings B (2024) Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning. In: Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics :15012\u201315032. https:\/\/doi.org\/10.18653\/v1\/2024.findings-emnlp.882","DOI":"10.18653\/v1\/2024.findings-emnlp.882"},{"key":"2449_CR44","doi-asserted-by":"publisher","unstructured":"Lanham T, Chen A, Radhakrishnan A et al (2023) Measuring Faithfulness in Chain-of-Thought Reasoning. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2307.13702","DOI":"10.48550\/arXiv.2307.13702"},{"key":"2449_CR45","doi-asserted-by":"publisher","unstructured":"Chen Y, Benton J, Radhakrishnan A et al (2025) Reasoning Models Don\u2019t Always Say What They Think. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2505.05410","DOI":"10.48550\/arXiv.2505.05410"},{"key":"2449_CR46","doi-asserted-by":"publisher","unstructured":"Parcalabescu L, Frank A (2024) On Measuring Faithfulness or Self-consistency of Natural Language Explanations. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics :6048\u20136089. https:\/\/doi.org\/10.18653\/v1\/2024.acl-long.329","DOI":"10.18653\/v1\/2024.acl-long.329"},{"issue":"5","key":"2449_CR47","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1016\/0895-4356(93)90018-V","volume":"46","author":"T Byrt","year":"1993","unstructured":"Byrt T, Bishop J, Carlin JB (1993) Bias, prevalence and kappa. J Clin Epidemiol 46(5):423\u2013429. https:\/\/doi.org\/10.1016\/0895-4356(93)90018-V","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"2449_CR48","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159\u2013174. https:\/\/doi.org\/10.2307\/2529310","journal-title":"Biometrics"},{"key":"2449_CR49","doi-asserted-by":"publisher","unstructured":"Jacovi A, Bitton Y, Bohnet B et al (2024) A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics :4615\u20134634. https:\/\/doi.org\/10.18653\/v1\/2024.acl-long.254","DOI":"10.18653\/v1\/2024.acl-long.254"},{"key":"2449_CR50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/1609406919899220","volume":"19","author":"C O\u2019Connor","year":"2020","unstructured":"O\u2019Connor C, Joffe H (2020) Intercoder reliability in qualitative research: debates and practical guidelines. Int J Qual Methods 19:1\u201313. https:\/\/doi.org\/10.1177\/1609406919899220","journal-title":"Int J Qual Methods"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-026-02449-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-026-02449-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-026-02449-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T06:52:04Z","timestamp":1786171924000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-026-02449-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,8]]},"references-count":50,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["2449"],"URL":"https:\/\/doi.org\/10.1007\/s10916-026-02449-3","relation":{},"ISSN":["1573-689X"],"issn-type":[{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,8]]},"assertion":[{"value":"18 May 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 July 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 August 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The study was approved by the Paracelsus Medical Private University ethics committee (PMU-EK-2025-0054). Analyses of public posts followed the Association of Internet Researchers (AoIR) Ethics 3.0 guidelines for internet research [\n                      \n                      ], with a focus on context-sensitive, platform-aware practices. We did not interact with individual users, nor did we process or retain deleted or protected content. To balance privacy with scientific validity, we only include verbatim tweet excerpts when necessary to demonstrate specific linguistic reasoning failures.","order":1,"name":"Ethics","label":"Ethics Approval","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":3,"name":"Ethics","label":"Consent to Participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":4,"name":"Ethics","label":"Consent to Publish","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":5,"name":"Ethics","label":"Clinical Trial Number","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"123"}}