{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T11:38:23Z","timestamp":1787312303525,"version":"build-2736575974"},"reference-count":51,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Pharmacoepidemiologic studies, which promote rational drug use and improve health outcomes, often require Anatomical Therapeutic Chemical Classification System (ATC) drug classification within real-world data (RWD) sources. Existing classification tools are expensive, brittle, or have restrictive terms of service, and lack context that may inform classification itself.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study sought to establish large language models (LLMs) as an assisting technology in the drug classification task. This included developing artificial intelligence prompts that reason about drugs using RWD and showing that the resulting accuracy, efficiency, and effectiveness are favorable to alternative methods.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A prompt was constructed to classify aspirin as either an analgesic or antithrombotic and evaluated within 12,294 anonymized daily dose strings from a polychronic population residing in the United States and Canada. The patients used a smart medication dispenser called \u201cspencer\u201d and consented to the use of their data for research. The LLM prompt requested that the best and next-best second-level ATC code be returned, and grading was performed on a 3-point scale. After success in a pilot sample of 20, an inference sample of 200 was taken without replacement. Finite population inference was carried out on the proportion of outputs receiving 1 of the top 2 grades. As a benchmark, Google\u2019s Programmable Search Engine was used to query the drug name plus \u201cATC code\u201d followed by regex-based extraction of ATC codes. All imperfect results were reviewed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The population consisted of 12,294 daily dose strings from 86.26% (2908\/3371) patients residing in Canada and 13.73% (463\/3371) residing in the United States. A prompt using the chain-of-thought reasoning was able to distinguish between aspirin\u2019s analgesic versus antithrombotic therapeutic uses and performed well in the pilot sample. In the inferential sample, 87.5% (175\/200) were graded as perfect, 5% (10\/200) had a minor issue, and 7.5% (15\/200) had a major issue. The estimate of the proportion of at least mostly correct classification was 92.5% (185\/200, 80% CI 90.1%-94.9%). For the search-based algorithm, 82.5% (165\/200) were deemed acceptable. The chain-of-thought reasoning was most helpful with supplements (eg, folic acid) when high doses indicated antianemic preparations. The problem formulation of daily dose inputs and multiple ATC outputs was sometimes incompatible with the drug (eg, pregabalin, calcitriol, and methotrexate).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>GPT-4o offers cost-effective drug classification from RWD without violating any terms of service. Using a chain-of-thought prompting technique, GPT-4o can reason about drug dosages that affect the class. The wide accessibility of LLMs gives every research team the ability to classify drugs at scale, a key prerequisite of pharmacoepidemiologic research.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/65481","type":"journal-article","created":{"date-parts":[[2025,4,14]],"date-time":"2025-04-14T09:50:06Z","timestamp":1744624206000},"page":"e65481","source":"Crossref","is-referenced-by-count":4,"title":["AI-Powered Drug Classification and Indication Mapping for Pharmacoepidemiologic Studies: Prompt Development and Validation"],"prefix":"10.2196","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5965-3097","authenticated-orcid":false,"given":"Benjamin","family":"Ogorek","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6609-2721","authenticated-orcid":false,"given":"Thomas","family":"Rhoads","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6339-6281","authenticated-orcid":false,"given":"Eric","family":"Finkelman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1787-8381","authenticated-orcid":false,"given":"Isaac R","family":"Rodriguez-Chavez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,6,12]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2021.723427"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2009.002477"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2004.10.012"},{"key":"ref4","first-page":"574","volume":"216","author":"Hripcsak, G","year":"2015","journal-title":"Stud Health Technol Inform"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMp0905338"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.13063\/2327-9214.1189"},{"key":"ref7","author":"Brown, TB","journal-title":"arXiv. Preprint posted online May 28, 2020"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.3390\/jcm12227033"},{"key":"ref9","author":"Williams, N","journal-title":"arXiv. Preprint posted online April 1, 2024"},{"key":"ref10","unstructured":"Applications of the ATC\/DDD methodologyWorld Health Organization2024-06-23https:\/\/www.who.int\/tools\/atc-ddd-toolkit\/applications-methodology"},{"key":"ref11","unstructured":"WHO Collaborating Centre for Drug Statistics Methodology. ATC\/DDD methodology \u2013 historyNorwegian Institute of Public Health20182024-07-16https:\/\/atcddd.fhi.no\/atc_ddd_methodology\/history\/"},{"key":"ref12","unstructured":"Anatomical therapeutic chemical (ATC) classificationWorld Health Organization2024-07-16https:\/\/www.who.int\/tools\/atc-ddd-toolkit\/atc-classification"},{"key":"ref13","unstructured":"WHO Collaborating Centre for Drug Statistics Methodology. ATC\/DDD Index \u2013 C09BA03Norwegian Institute of Public Health2024-07-16https:\/\/atcddd.fhi.no\/atc_ddd_index\/?code=C09BA03"},{"key":"ref14","unstructured":"WHO Collaborating Centre for Drug Statistics Methodology. Purpose of the ATC\/DDD systemNorwegian Institute of Public Health20182024-07-16https:\/\/atcddd.fhi.no\/atc_ddd_methodology\/purpose_of_the_atc_ddd_system\/"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.2146\/ajhp080221"},{"key":"ref16","unstructured":"Drug identification number (DIN)Health Canada2024-06-23https:\/\/www.canada.ca\/en\/health-canada\/services\/drugs-health-products\/drug-products\/fact-sheets\/drug-identification-number.html"},{"key":"ref17","unstructured":"About Z-indexZ-Index2024-07-18https:\/\/www.z-index.nl\/english"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-76298-0_52"},{"key":"ref19","unstructured":"DBpedia lookup - generic RDF indexer and searcherDBpedia2024-12-09https:\/\/lookup.dbpedia.org\/"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1109\/mitp.2005.122"},{"key":"ref21","unstructured":"RxclassUnited States National Library of Medicine2024-06-23https:\/\/mor.nlm.nih.gov\/RxClass\/"},{"key":"ref22","unstructured":"Drug product database: access the extractsHealth Canada2022-04-30https:\/\/www.canada.ca\/en\/health-canada\/services\/drugs-health-products\/drug-products\/drug-product-database\/extracts.html"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1093\/database\/baad019"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.32388\/0gv0fn"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkad976"},{"key":"ref26","unstructured":"About DrugBankDrugBank2024-06-13https:\/\/go.drugbank.com\/about"},{"key":"ref27","unstructured":"Custom search JSON APIGoogle2024-12-09https:\/\/developers.google.com\/custom-search\/v1\/overview"},{"key":"ref28","unstructured":"WallaceJCBerzonMShiQ Labs V. Linkedln CorpUnited States Supreme Court2024-04-29https:\/\/cdn.ca9.uscourts.gov\/datastore\/opinions\/2022\/04\/18\/17-16783.pdf"},{"key":"ref29","unstructured":"Programmable search engine terms of serviceGoogle2024-12-09https:\/\/support.google.com\/programmable-search\/answer\/1714300?hl=en"},{"issue":"1","key":"ref30","first-page":"28","volume":"301","author":"Liu, HW","year":"2020","journal-title":"Wash Int Law J"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.2196\/34906"},{"key":"ref32","unstructured":"GPT-4oOpenAI2024-06-24https:\/\/platform.openai.com\/docs\/models\/gpt-4o"},{"key":"ref33","unstructured":"Batch APIOpen AI2024-07-01https:\/\/platform.openai.com\/docs\/guides\/batch"},{"key":"ref34","unstructured":"How should I set the temperature parameter?OpenAI2024-07-28https:\/\/platform.openai.com\/docs\/guides\/text-generation\/how-should-i-set-the-temperature-parameter"},{"key":"ref35","unstructured":"max_tokensOpenAI2024-07-28https:\/\/platform.openai.com\/docs\/api-reference\/chat\/create#chat-create-max_tokens"},{"key":"ref36","unstructured":"What are tokens and how to count them?OpenAI2024-07-28https:\/\/help.openai.com\/en\/articles\/4936856-what-are-tokens-and-how-to-count-them"},{"key":"ref37","doi-asserted-by":"crossref","DOI":"10.1201\/9780429298899","author":"Lohr, SL","year":"2021","journal-title":"Sampling: Design and Analysis"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2012.09.002"},{"key":"ref39","unstructured":"Canadian Institutes of Health ResearchNatural Sciences and Engineering Research Council of CanadaSocial Sciences and Humanities Research Council of CanadaTri-Council Policy Statement: Ethical Conduct for Research Involving HumansPanel on Research Ethics20182025-05-22https:\/\/ethics.gc.ca\/eng\/documents\/tcps2-2018-en-interactive-final.pdf"},{"key":"ref40","unstructured":"US Department of Health and Human Services45 CFR \u00a746.104(d)(4)(ii): Exempt researchCode of Federal Regulations20182022-05-22https:\/\/www.ecfr.gov\/current\/title-45\/subtitle-A\/subchapter-A\/part-46#section-46.104"},{"key":"ref41","unstructured":"Product monograph PrMYA\u00aeAPOTEX INC2024-04-29https:\/\/pdf.hres.ca\/dpd_pm\/00055884.PDF"},{"key":"ref42","unstructured":"STATEX\u00aePaladin Labs Inc2024-08-05https:\/\/www.paladin-pharma.com\/our_products\/Statex_en.pdf"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-56063-7_4"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1126\/science.1248506"},{"key":"ref45","unstructured":"Why the API output is inconsistent even after the temperature is set to 0OpenAI Community2024-07-24https:\/\/community.openai.com\/t\/why-the-api-output-is-inconsistent-even-after-the-temperature-is-set-to-0\/329541"},{"key":"ref46","author":"Wang, X","journal-title":"arXiv. Preprint posted online March 21, 2022"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1101\/2024.11.19.24317555"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.3389\/fvets.2024.1490030"},{"key":"ref49","unstructured":"Hello GPT-4oOpenAI2024-07-24https:\/\/openai.com\/index\/hello-gpt-4o\/"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.9.3.458"},{"key":"ref51","author":"Touvron, H","journal-title":"arXiv. Preprint posted online February 27, 2023"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e65481","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T13:58:32Z","timestamp":1785851912000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e65481"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,12]]},"references-count":51,"URL":"https:\/\/doi.org\/10.2196\/65481","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.65481","asserted-by":"object"}]},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,12]]}}}