{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,11]],"date-time":"2026-11-11T16:29:18Z","timestamp":1794414558526,"version":"4.9.1"},"reference-count":33,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Diagnostic errors in primary care contribute substantially to avoidable morbidity. Real-time, voice-based artificial intelligence (AI) diagnostic assistants may support diagnostic reasoning during clinical encounters, but formative evidence that can be used to assess both diagnostic benefit and overreliance risk before clinical deployment is needed from interactive settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to conduct a formative, high-difficulty simulation stress test of whether a real-time, voice-based AI diagnostic assistant was associated with improved physician diagnostic accuracy and measurable overreliance during simulated primary care consultations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a case-level randomized, adjudicator-blinded simulation study in a web-based virtual primary care clinic. Thirteen board-certified family and community medicine physicians managed 260 simulated voice-based consultations. Cases were assigned to either an AI-assisted condition or an unassisted, resource-restricted simulation condition without any external diagnostic aids. No real patients were enrolled, and no clinical care was delivered or modified. The primary outcome was Top-3 diagnostic accuracy, defined as at least one correct diagnosis among up to 3 submitted diagnoses assessed by 3 blinded adjudicators and analyzed using a frequentist binomial generalized linear mixed model fitted by maximum likelihood with random intercepts for physician and case.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      All 260 consultations were completed and analyzed. Adjusted Top-3 diagnostic accuracy was 62.3% (95% CI 49.1% to 75.7%) in the unassisted condition and 74.6% (95% CI 63.7% to 85.4%) in the AI-assisted condition (adjusted odds ratio [AOR] 2.68, 95% CI 1.24 to 6.12; \u03c7\n                      <jats:sup>2<\/jats:sup>\n                      <jats:sub>1<\/jats:sub>\n                      =6.4,\n                      <jats:italic>P<\/jats:italic>\n                      =.01). This corresponds to an adjusted absolute difference of +12.3 percentage points (95% CI +2.7 to +22.6) and a simulation-context number needed to treat equivalent of 8.1 (95% CI 4.4 to 37.2). The post hoc Top-2 analysis was supportive (AOR 2.59, 95% CI 1.23 to 5.77; \u03c7\n                      <jats:sup>2<\/jats:sup>\n                      <jats:sub>1<\/jats:sub>\n                      =6.3,\n                      <jats:italic>P<\/jats:italic>\n                      =.01). Top-1 point estimates favored AI assistance but were inconclusive: Adjusted accuracy was 47.8% in the unassisted condition versus 57% with AI assistance (AOR 1.83, 95% CI 0.93 to 3.80; \u03c7\n                      <jats:sup>2<\/jats:sup>\n                      <jats:sub>1<\/jats:sub>\n                      =3.1,\n                      <jats:italic>P<\/jats:italic>\n                      =.08). Leave-one-physician-out and virtual-patient simulator-exclusion sensitivity analyses were consistent with the primary finding. In risk scenarios with an incorrect operative AI suggestion, adjusted concordance was 38.8% in the unassisted condition and 55.8% in the AI-assisted condition, but the between-condition comparison was not statistically significant (AOR 2.08, 95% CI 0.89 to 5.19; \u03c7\n                      <jats:sup>2<\/jats:sup>\n                      <jats:sub>1<\/jats:sub>\n                      =2.9,\n                      <jats:italic>P<\/jats:italic>\n                      =.09). Mean consultation time increased by 10.7%, and physicians rated the system highly for usefulness and satisfaction.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>In this randomized, high-difficulty simulation study, real-time, voice-based AI assistance was associated with higher physician Top-3 diagnostic accuracy than an unassisted, resource-restricted comparator. Some sensitivity analyses were supportive, while others were directionally favorable but inconclusive, and the safety analysis suggested possible overreliance without statistically conclusive difference between conditions. These findings support further refinement and prospective evaluation in real clinical workflows before conclusions are drawn about routine clinical effectiveness or implementation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/104579","type":"journal-article","created":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T14:16:20Z","timestamp":1788444980000},"page":"e104579-e104579","source":"Crossref","is-referenced-by-count":1,"title":["Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4984-0639","authenticated-orcid":false,"given":"Ivan","family":"Cusacovich","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-6462-6139","authenticated-orcid":false,"given":"Manuel","family":"Pinilla","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8299-1706","authenticated-orcid":false,"given":"Ruben","family":"Garcia Castro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"issue":"9","key":"R1","doi-asserted-by":"publisher","first-page":"727","DOI":"10.1136\/bmjqs-2013-002627","article-title":"The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations","volume":"23","author":"Singh","journal-title":"BMJ Qual Saf"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1136\/bmjqs-2016-005401","article-title":"The global burden of diagnostic errors in primary care","volume":"26","author":"Singh","journal-title":"BMJ Qual Saf"},{"issue":"6","key":"R3","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1093\/fampra\/cmn071","article-title":"Diagnostic difficulty and error in primary care--a systematic review","volume":"25","author":"Kostopoulou","journal-title":"Fam Pract"},{"issue":"3","key":"R4","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2016-012789","article-title":"Clinical characteristics associated with diagnostic delay of pulmonary embolism in primary care: a retrospective observational study","volume":"7","author":"Hendriksen","journal-title":"BMJ Open"},{"issue":"11","key":"R5","doi-asserted-by":"publisher","first-page":"4819","DOI":"10.4103\/jfmpc.jfmpc_1177_24","article-title":"Understanding the impact of delayed diagnosis and misdiagnosis of systemic lupus erythematosus (SLE)","volume":"13","author":"Mitchell","journal-title":"J Family Med Prim Care"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","DOI":"10.1038\/s44387-025-00011-z","article-title":"Large language models for disease diagnosis: a scoping review","volume":"1","author":"Zhou","journal-title":"NPJ Artif Intell"},{"issue":"8067","key":"R7","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1038\/s41586-025-08866-7","article-title":"Towards conversational diagnostic artificial intelligence","volume":"642","author":"Tu","journal-title":"Nature"},{"issue":"8067","key":"R8","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1038\/s41586-025-08869-4","article-title":"Towards accurate differential diagnosis with large language models","volume":"642","author":"McDuff","journal-title":"Nature"},{"issue":"10","key":"R9","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large language model influence on diagnostic reasoning","volume":"7","author":"Goh","journal-title":"JAMA Netw Open"},{"issue":"4","key":"R10","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1038\/s41591-024-03456-y","article-title":"GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial","volume":"31","author":"Goh","journal-title":"Nat Med"},{"issue":"2","key":"R11","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1038\/s44360-025-00007-8","article-title":"Large language model diagnostic assistance for physicians in a lower-middle-income country: a randomized controlled trial","volume":"1","author":"Qazi","journal-title":"Nat Health"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-026-02545-1","article-title":"From tool to teammate in a randomized controlled trial of clinician-AI collaborative workflows for diagnosis","volume":"9","author":"Everett","journal-title":"NPJ Digit Med"},{"issue":"13","key":"R13","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","journal-title":"N Engl J Med"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"e12","DOI":"10.1016\/S2589-7500(23)00225-X","article-title":"Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study","volume":"6","author":"Zack","journal-title":"Lancet Digit Health"},{"issue":"4","key":"R15","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1007\/s11606-024-09177-9","article-title":"Bias sensitivity in diagnostic decision-making: comparing ChatGPT with residents","volume":"40","author":"Schmidt","journal-title":"J GEN INTERN MED"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0232-8","article-title":"Impact of a deep learning assistant on the histopathologic classification of liver cancer","volume":"3","author":"Kiani","journal-title":"NPJ Digit Med"},{"issue":"10","key":"R17","doi-asserted-by":"publisher","first-page":"1684","DOI":"10.1093\/jamia\/ocad118","article-title":"Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias","volume":"30","author":"Wang","journal-title":"J Am Med Inform Assoc"},{"issue":"23","key":"R18","doi-asserted-by":"publisher","first-page":"2275","DOI":"10.1001\/jama.2023.22295","article-title":"Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study","volume":"330","author":"Jabbour","journal-title":"JAMA"},{"issue":"5","key":"R19","doi-asserted-by":"publisher","first-page":"1224","DOI":"10.1002\/acn3.52036","article-title":"Effects of explainable artificial intelligence in neurology decision support","volume":"11","author":"Gombolay","journal-title":"Ann Clin Transl Neurol"},{"issue":"17","key":"R20","doi-asserted-by":"publisher","first-page":"2154","DOI":"10.3390\/healthcare13172154","article-title":"Explainable AI in clinical decision support systems: a meta-analysis of methods, applications, and usability challenges","volume":"13","author":"Abbas","journal-title":"Healthcare"},{"issue":"5","key":"R21","doi-asserted-by":"publisher","DOI":"10.1056\/CAT.25.0040","article-title":"Ambient artificial intelligence scribes: learnings after 1 year and over 2.5 million uses","volume":"6","author":"Tierney","journal-title":"NEJM Catalyst"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1093\/jamia\/ocae304","article-title":"Ambient artificial intelligence scribes: utilization and impact on documentation time","volume":"32","author":"Ma","journal-title":"J Am Med Inform Assoc"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.2196\/44773","article-title":"Standardized comparison of voice-based information and documentation systems to established systems in intensive care: crossover study","volume":"11","author":"Peine","journal-title":"JMIR Med Inform"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104483","article-title":"A voice-based digital assistant for intelligent prompting of evidence-based practices during ICU rounds","volume":"146","author":"King","journal-title":"J Biomed Inform"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","DOI":"10.2196\/53961","article-title":"A generative pretrained transformer (GPT)-powered chatbot as a simulated patient to practice history taking: prospective, mixed methods study","volume":"10","author":"Holderried","journal-title":"JMIR Med Educ"},{"issue":"1","key":"R26","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-025-01283-x","article-title":"Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education","volume":"6","author":"Yu","journal-title":"Commun Med (Lond)"},{"key":"R27","doi-asserted-by":"crossref","unstructured":"Cusacovich I Pinilla M Garcia Castro R . A real-time artificial intelligence diagnostic copilot in simulated primary care consultations: randomized simulation study. Zenodo. 2026. URL: https:\/\/zenodo.org\/records\/22031287 [Accessed 12-09-2026]","DOI":"10.2196\/104579"},{"key":"R28","unstructured":"European Union. Reglamento (UE) 2016\/679 del parlamento europeo y del consejo de 27 de abril de 2016 relativo a la protecci\u00f3n de las personas f\u00edsicas en lo que respecta al tratamiento de datos personales. Agencia Estatal Bolet\u00edn Oficial del Estado; Apr 27, 2016. URL: https:\/\/www.boe.es\/doue\/2016\/119\/L00001-00088.pdf [Accessed 10-09-2026]"},{"key":"R29","unstructured":"Kingdom of Spain. Ley Org\u00e1nica 3\/2018, de 5 de diciembre, de Protecci\u00f3n de Datos Personales y garant\u00eda de los derechos digitales. Agencia Estatal Bolet\u00edn Oficial del Estado; Dec 6, 2018. URL: https:\/\/www.boe.es\/eli\/es\/lo\/2018\/12\/05\/3\/con [Accessed 10-09-2026]"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.2196\/46875","article-title":"Evaluating the diagnostic performance of symptom checkers: clinical vignette study","volume":"3","author":"Hammoud","journal-title":"JMIR AI"},{"issue":"5","key":"R31","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2501001","article-title":"Automation bias in large language model\u2013assisted diagnostic reasoning among physicians trained in AI literacy \u2014 a randomized clinical trial","volume":"3","author":"Qazi","journal-title":"NEJM AI"},{"issue":"3","key":"R32","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1177\/0018720810376055","article-title":"Complacency and bias in human use of automation: an attentional integration","volume":"52","author":"Parasuraman","journal-title":"Hum Factors"},{"issue":"4","key":"R33","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1006\/ijhc.1999.0349","article-title":"Accountability and automation bias","volume":"52","author":"Skitka","journal-title":"Int J Hum Comput Stud"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e104579","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,11,11]],"date-time":"2026-11-11T16:16:22Z","timestamp":1794413782000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e104579"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":33,"URL":"https:\/\/doi.org\/10.2196\/104579","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]},"article-number":"v10i9e104579"}}