{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,12]],"date-time":"2026-11-12T08:34:19Z","timestamp":1794472459122,"version":"4.9.1"},"reference-count":36,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"unspecified","delay-in-days":6,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Digit Health"],"accepted":{"date-parts":[[2026,7,13]]},"published-print":{"date-parts":[[2026,10]]},"abstract":"<jats:p>The flexibility of generative artificial intelligence (AI), and specifically large language models (LLMs), holds promise to transform healthcare by offering new tools to improve patient care and enhance clinician efficiency. However, this flexibility introduces new challenges, as LLM applications might inadvertently be used in ways that do not align with the intended purpose of the manufacturer, raising patient safety concerns and liability issues. We have identified a regulatory gap whereby LLM applications that could be repurposed, malfunction or be inadvertently misused for medical purposes fall outside the scope of the Medical Device Regulation (MDR) because they lack a medical intended purpose, and outside meaningful AI Act safety obligations because they do not qualify as high-risk systems, yet still present patient safety risks without adequate regulatory guidance. In this manuscript, we aim to address this regulatory gap for LLM applications in healthcare by identifying these ambiguous grey-area applications and providing four recommendations\u2014validation, mitigation, compliance and alignment\u2014focused on prevention and risk mitigation rather than reclassification as medical devices. These recommendations are intended to advance the responsible implementation of LLM applications in healthcare while ensuring patient safety.<\/jats:p>","DOI":"10.1136\/bmjdh-2026-000003","type":"journal-article","created":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T14:55:16Z","timestamp":1791384916000},"page":"e000003","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":2,"title":["When are large language models medical devices? Navigating the MDR and AI Act for healthcare applications"],"prefix":"10.1136","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4134-0242","authenticated-orcid":false,"given":"Jessica D","family":"Workum","sequence":"first","affiliation":[{"name":"Department of Adult Intensive Care, Erasmus MC, Rotterdam, The Netherlands"},{"name":"Elisabeth-TweeSteden Hospital, Tilburg, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sade J","family":"Krijgsman","sequence":"additional","affiliation":[{"name":"Department of Clinical Physics and Informatics, Maasstad Hospital, Rotterdam, The Netherlands"},{"name":"Romion Health, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ildiko","family":"Vajda","sequence":"additional","affiliation":[{"name":"Dutch Patients\u2019 Federation, Utrecht, The Netherlands"},{"name":"Working Group on Digital Health, European Patients\u2019 Forum, Brussels, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"R M","family":"Van Stokkum","sequence":"additional","affiliation":[{"name":"Department of Risk Analysis for Products in Development, Netherlands Organization for Applied Scientific Research, The Hague, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan","family":"van de Brand","sequence":"additional","affiliation":[{"name":"Department of Adult Intensive Care, Erasmus MC, Rotterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diederik","family":"Gommers","sequence":"additional","affiliation":[{"name":"Department of Adult Intensive Care, Erasmus MC, Rotterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8350-8093","authenticated-orcid":false,"given":"Ari","family":"Ercole","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Cambridge, Cambridge, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5668-3435","authenticated-orcid":false,"given":"Michel","family":"van Genderen","sequence":"additional","affiliation":[{"name":"Department of Adult Intensive Care, Erasmus MC, Rotterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"key":"2026100707550955000_2.1.e000003.1","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2025-099301","article-title":"Quality and efficiency of integrating customised large language model-generated summaries versus physician-written summaries: a validation study","volume":"15","author":"Schoonbeek","year":"2025","journal-title":"BMJ Open"},{"key":"2026100707550955000_2.1.e000003.2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"2026100707550955000_2.1.e000003.3","doi-asserted-by":"crossref","DOI":"10.2196\/52865","article-title":"The Impact of Multimodal Large Language Models on Health Care\u2019s Future","volume":"25","author":"Mesk\u00f3","year":"2023","journal-title":"J Med Internet Res"},{"key":"2026100707550955000_2.1.e000003.4","unstructured":"European Parliament and Council of the European Union . Regulation (EU) 2017\/745 of the european parliament and of the council of 5 april 2017 on medical devices, amending directive 2001\/83\/EC. Official Journal of the European Union; 2017. Available: http:\/\/data.europa.eu\/eli\/reg\/2017\/745\/2025-01-10"},{"key":"2026100707550955000_2.1.e000003.5","doi-asserted-by":"crossref","unstructured":"Blumenthal D , Patel B . The Regulation of Clinical Artificial Intelligence. NEJM AI 2024;1. doi:10.1056\/AIpc2400545","DOI":"10.1056\/AIpc2400545"},{"key":"2026100707550955000_2.1.e000003.6","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-023-00873-0","article-title":"The imperative for regulatory oversight of large language models (or generative AI) in healthcare","volume":"6","author":"Mesk\u00f3","year":"2023","journal-title":"NPJ Digit Med"},{"key":"2026100707550955000_2.1.e000003.7","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-025-01544-y","article-title":"Unregulated large language models produce medical device-like output","volume":"8","author":"Weissman","year":"2025","journal-title":"NPJ Digit Med"},{"key":"2026100707550955000_2.1.e000003.8","doi-asserted-by":"crossref","first-page":"e662","DOI":"10.1016\/S2589-7500(24)00124-9","article-title":"A future role for health applications of large language models depends on regulators enforcing safety standards","volume":"6","author":"Freyer","year":"2024","journal-title":"Lancet Digit Health"},{"key":"2026100707550955000_2.1.e000003.9","unstructured":"European Parliament and Council of the European Union . Official Journal of the European Union; Regulation (EU) 2024\/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations, 2024. Available: http:\/\/data.europa.eu\/eli\/reg\/2024\/1689\/oj"},{"key":"2026100707550955000_2.1.e000003.10","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-024-01213-6","article-title":"Navigating the European Union Artificial Intelligence Act for Healthcare","volume":"7","author":"Busch","year":"2024","journal-title":"NPJ Digit Med"},{"key":"2026100707550955000_2.1.e000003.11","doi-asserted-by":"crossref","unstructured":"Porsdam Mann S , Cohen IG , Minssen T . The EU AI Act: Implications for U.S. Health Care. NEJM AI 2024;1. doi:10.1056\/AIp2400449","DOI":"10.1056\/AIp2400449"},{"key":"2026100707550955000_2.1.e000003.12","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-024-01232-3","article-title":"Navigating the EU AI Act: implications for regulated digital medical products","volume":"7","author":"Aboy","year":"2024","journal-title":"NPJ Digit Med"},{"key":"2026100707550955000_2.1.e000003.13","unstructured":"Medical Device Coordination Group (MDCG) . MDCG 2025-6: Interplay between the Medical Devices Regulation (MDR) & In Vitro Diagnostic Medical Devices Regulation (IVDR) and the Artificial Intelligence Act (AIA). 2025."},{"key":"2026100707550955000_2.1.e000003.14","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2023.9651"},{"key":"2026100707550955000_2.1.e000003.15","doi-asserted-by":"crossref","first-page":"2396","DOI":"10.1038\/s41591-023-02412-6","article-title":"Large language model AI chatbots require approval as medical devices","volume":"29","author":"Gilbert","year":"2023","journal-title":"Nat Med"},{"key":"2026100707550955000_2.1.e000003.16","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1017\/jme.2025.10136","article-title":"Do Specialized Medical LLMs Demand a Radically New Approach Under the EU\u2019s Medical Device Regulation?","volume":"53","author":"Smith","year":"2025","journal-title":"J Law Med Ethics"},{"key":"2026100707550955000_2.1.e000003.17","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-025-01800-1","article-title":"How AI is used in FDA-authorized medical devices: a taxonomy across 1,016 authorizations","volume":"8","author":"Singh","year":"2025","journal-title":"NPJ Digit Med"},{"key":"2026100707550955000_2.1.e000003.18","unstructured":"Medical Device Coordination Group (MDCG) . MDCG 2019-11 rev 1. guidance on qualification and classification of software in regulation (EU) 2017\/745 \u2013 MDR and regulation (EU) 2017\/746 \u2013 IVDR. 2019. Available: https:\/\/ec.europa.eu\/docsroom\/documents\/37581 [Accessed 13 Jul 2025]."},{"key":"2026100707550955000_2.1.e000003.19","unstructured":"Medical Device Coordination Group (MDCG) . MDCG 2021-24: guidance on classification of medical devices. 2021."},{"key":"2026100707550955000_2.1.e000003.20","doi-asserted-by":"crossref","unstructured":"Ohde JW , Rost LM , Overgaard JD . The Burden of Reviewing LLM-Generated Content. NEJM AI 2025;2. doi:10.1056\/AIp2400979","DOI":"10.1056\/AIp2400979"},{"key":"2026100707550955000_2.1.e000003.21","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnlssr.2024.06.001"},{"key":"2026100707550955000_2.1.e000003.22","unstructured":"MedTech Europe . Medical technology industry perspective on the final AI Act, 2024. Available: www.medtecheurope.org"},{"key":"2026100707550955000_2.1.e000003.23","doi-asserted-by":"crossref","DOI":"10.3389\/frai.2025.1504805","article-title":"Bridging the gap: a practical step-by-step approach to warrant safe implementation of large language models in healthcare","volume":"8","author":"Workum","year":"2025","journal-title":"Front Artif Intell"},{"key":"2026100707550955000_2.1.e000003.24","unstructured":"Engelfriet A . Amsterdam: ICTRecht\u202f; Ius Mentis; The annotated AI act: article-by-article analysis of European AI legislation, 2024. Available: https:\/\/www.managementboek.nl\/boek\/9789083095769\/the-annotated-ai-act-2024-edition-arnoud-engelfriet [Accessed 20 Mar 2025]."},{"key":"2026100707550955000_2.1.e000003.25","unstructured":"Workum JD . RIGH:T: development of a validation framework for generative AI systems in healthcare, 2025. Available: https:\/\/www.zonmw.nl\/nl\/artikel\/right-ontwikkeling-van-een-validatieframework-voor-generatieve-ai-systemen-de-zorg [Accessed 16 Nov 2025]."},{"key":"2026100707550955000_2.1.e000003.26","unstructured":"Health and social care act 2008 (regulated activities) regulations 2014: regulation 17. United Kingdom; 2014. Available: https:\/\/www.cqc.org.uk\/guidance-regulation\/providers\/regulations-service-providers-and-managers\/health-social-care-act\/regulation-17 [Accessed 29 Sep 2025]."},{"key":"2026100707550955000_2.1.e000003.27","unstructured":"Price-Forbes A . Priority notification: ensuring safe and assured adoption of ai scribe technology. 2025."},{"key":"2026100707550955000_2.1.e000003.28","unstructured":"NHS England . Guidance on the use of AI-enabled ambient scribing products in health and care settings, Available: https:\/\/www.england.nhs.uk\/long-read\/guidance-on-the-use-of-ai-enabled-ambient-scribing-products-in-health-and-care-settings\/ [Accessed 2 Jul 2025]."},{"key":"2026100707550955000_2.1.e000003.29","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.r1302","article-title":"Doctors must stop using unregistered AI scribe tools, says NHS England","volume":"389","author":"Moberly","year":"2025","journal-title":"BMJ"},{"key":"2026100707550955000_2.1.e000003.30","unstructured":"Medicines and Healthcare Products Regulatory Agency. Ambient Voice Technology-Enabled Products. GOV.UK, 2026. Available: https:\/\/www.gov.uk\/government\/publications\/ambient-voice-technology-enabled-products\/ambient-voice-technology-enabled-products [Accessed 29 Jul 2026]."},{"key":"2026100707550955000_2.1.e000003.31","unstructured":"Digital scribes . Therapeutic goods administration (TGA). Available: https:\/\/www.tga.gov.au\/how-we-regulate\/manufacturing\/manufacture-medical-device\/manufacture-specific-types-medical-devices\/artificial-intelligence-ai-and-medical-device-software\/digital-scribes [Accessed 22 Aug 2025]."},{"key":"2026100707550955000_2.1.e000003.32","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2024.21451"},{"key":"2026100707550955000_2.1.e000003.33","unstructured":"U.s. department of health and human services food and drug administration. clinical decision support software. Available: https:\/\/www.fda.gov\/drugs\/guidance"},{"key":"2026100707550955000_2.1.e000003.34","unstructured":"Department of health and human services food and drug administration . Changes to existing medical software policies resulting from section 3060 of the 21st century cures act. 2019. Available: https:\/\/www.fda.gov\/vaccines-blood"},{"key":"2026100707550955000_2.1.e000003.35","unstructured":"European Commission . Proposal for a regulation of the European parliament and of the council amending regulations (EU) 2024\/1689 and (EU) 2018\/1139 as regards the simplification of the implementation of harmonised rules on artificial intelligence (digital omnibus on AI). 2026. Available: https:\/\/ai-act-service-desk.ec.europa.eu\/"},{"key":"2026100707550955000_2.1.e000003.36","unstructured":"European Commission . Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Regulations (EU) 2017\/745 and (EU) 2017\/746 as regards simplifying and reducing the burden of the rules on medical devices and in vitro diagnostic medical devices, and amend, 2026. Available: https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=CELEX:52025PC1023 [Accessed 26 Apr 2026]."}],"container-title":["BMJ Digital Health &amp; AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjdh-2026-000003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T14:55:20Z","timestamp":1791384920000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjdigitalhealth.bmj.com\/lookup\/doi\/10.1136\/bmjdh-2026-000003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,10,7]]},"published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1136\/bmjdh-2026-000003"],"URL":"https:\/\/doi.org\/10.1136\/bmjdh-2026-000003","relation":{"has-part":[{"id-type":"doi","id":"https:\/\/doi.org\/10.1136\/bmjdh-2026-000003.supp1","asserted-by":"subject"}]},"ISSN":["3049-575X"],"issn-type":[{"value":"3049-575X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10]]}}}