{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T20:17:13Z","timestamp":1789503433029,"version":"build-2803163510"},"reference-count":50,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Large-scale international real-world evidence generation benefits from terminology harmonization. Despite widespread adoption of standardized vocabularies, their effective use and long-term sustainability at scale remain poorly understood.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to examine real-world terminology and code use in data across a federated network of observational data sources.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We conducted a 2-part survey of researchers and data owners within the Observational Health Data Sciences and Informatics community on their terminology use, challenges, and needs, accompanied by the analysis of code use across a subset of real-world data sources.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The survey covered 144 institutions across the United States, the United Kingdom, Europe, Asia, and Africa. Data on terminology use covered 60 sources, including 22 data sources that provided detailed code-level use information. We observed significant variations in terminology use, with 61 out of 89 terminologies used in the data present in less than 10% of the data sources. Code use even after data harmonization was also highly variable: less than 1% (95\/742,337) of codes were found in all data sources. Mapping and hierarchy completeness, terminology coverage, versioning, and terminology changes were among the most common challenges. We outlined several of our subsequent process improvements: community contribution and stewardship pipelines, metadata for relationships, and informatics tools for assessment of the impact of terminology change.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Terminology and coding inconsistencies across observational data sources require a standardized terminology system. Such a system is complex and time-consuming and needs community contribution and informatics solutions for harmonization to be scalable and sustainable. Even with a common reference standard, high heterogeneity of terminology and code use across different observational data sources remains.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/92727","type":"journal-article","created":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T19:43:56Z","timestamp":1789501436000},"page":"e92727-e92727","source":"Crossref","is-referenced-by-count":0,"title":["Real-World Use of Controlled Terminologies, Ontologies, and Vocabularies for Evidence Generation Across a Large International Observational Network: Challenges and Lessons Learned From a Mixed Method Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0847-6682","authenticated-orcid":false,"given":"Anna","family":"Ostropolets","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0543-4815","authenticated-orcid":false,"given":"Vlad","family":"Korsik","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4591-6085","authenticated-orcid":false,"given":"Tatsiana","family":"Skuhareuskaya","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9016-7602","authenticated-orcid":false,"given":"Aleh","family":"Zhuk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-6232-2311","authenticated-orcid":false,"given":"Maryia","family":"Khitrun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9978-4978","authenticated-orcid":false,"given":"Alexander","family":"Davydov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8718-0013","authenticated-orcid":false,"given":"Dmitry","family":"Dymshyts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3641-055X","authenticated-orcid":false,"given":"Christian","family":"Reich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2664-7614","authenticated-orcid":false,"given":"George","family":"Hripcsak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9727-2138","authenticated-orcid":false,"given":"Patrick","family":"Ryan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,15]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1055\/s-0041-1726481","article-title":"Drawing reproducible conclusions from observational clinical data with OHDSI","volume":"30","author":"Hripcsak","journal-title":"Yearb Med Inform"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-016-1615-z","article-title":"The effectiveness of clinical networks in improving quality of care and patient outcomes: a systematic review of quantitative and qualitative studies","volume":"16","author":"Brown","journal-title":"BMC Health Serv Res"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-022-01768-6","article-title":"Real-world data: a brief review of the methods, applications, challenges and opportunities","volume":"22","author":"Liu","journal-title":"BMC Med Res Methodol"},{"issue":"4","key":"R4","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1002\/cpt.1577","article-title":"Choosing among common data models for real-world data analyses fit for making decisions about the effectiveness of medical products","volume":"107","author":"Schneeweiss","journal-title":"Clin Pharmacol Ther"},{"issue":"3","key":"R5","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1093\/jamia\/ocad247","article-title":"OHDSI Standardized Vocabularies-a large-scale centralized reference ontology for international data harmonization","volume":"31","author":"Reich","journal-title":"J Am Med Inform Assoc"},{"key":"R6","volume":"216","author":"Hripcsak","journal-title":"Stud Health Technol Inform"},{"issue":"2","key":"R7","doi-asserted-by":"publisher","first-page":"309","DOI":"10.14236\/jhi.v22i2.159","article-title":"Using ontologies to improve semantic interoperability in health data","volume":"22","author":"Liyanage","journal-title":"J Innov Health Inform"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1055\/s-0038-1667077","article-title":"Recent developments in clinical terminologies - SNOMED CT, LOINC, and RxNorm","volume":"27","author":"Bodenreider","journal-title":"Yearb Med Inform"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2017.12.010","article-title":"Auditing SNOMED CT hierarchical relations based on lexical features of concepts in non-lattice subgraphs","volume":"78","author":"Cui","journal-title":"J Biomed Inform"},{"key":"R10","volume":"2024","author":"Hu","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"issue":"4","key":"R11","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1136\/amiajnl-2010-000045","article-title":"Getting the foot out of the pelvis: modeling problems affecting use of SNOMED CT hierarchies in practical applications","volume":"18","author":"Rector","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.13063\/2327-9214.1244","article-title":"A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data","volume":"4","author":"Kahn","journal-title":"EGEMS (Wash DC)"},{"key":"R13","unstructured":"Ostropolets A . ohdsi-studies\/ConceptPrevalence. GitHub. URL: https:\/\/github.com\/ohdsi-studies\/ConceptPrevalence [Accessed 30-08-2026]"},{"key":"R14","unstructured":"Athena. URL: https:\/\/athena.ohdsi.org\/search-terms\/start [Accessed 07-09-2026]"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1093\/jamia\/ocad214","article-title":"European Health Data & Evidence Network-learnings from building out a standardized international health data network","volume":"31","author":"Voss","journal-title":"J Am Med Inform Assoc"},{"key":"R16","unstructured":"Release planning. GitHub. URL: https:\/\/github.com\/OHDSI\/Vocabulary-v5.0\/wiki\/Release-planning [Accessed 11-11-2024]"},{"key":"R17","unstructured":"Upcoming changes. GitHub. URL: https:\/\/github.com\/OHDSI\/Vocabulary-v5.0\/wiki\/Upcoming-changes [Accessed 11-11-2024]"},{"key":"R18","unstructured":"Releases \u00b7 OHDSI\/Vocabulary-v5.0. GitHub. URL: https:\/\/github.com\/OHDSI\/Vocabulary-v5.0\/releases [Accessed 16-03-2023]"},{"key":"R19","unstructured":"Dymshyts D DeFalco F Ostropolets A Rao G Shoaibi A Blacketer C . Evaluating the impact of different vocabulary versions on cohort definitions and CDM. Observational Health Data Sciences and Informatics; 2024. URL: https:\/\/www.ohdsi.org\/2024showcase-23\/ [Accessed 30-08-2026]"},{"key":"R20","unstructured":"Community contribution. GitHub. URL: https:\/\/github.com\/OHDSI\/Vocabulary-v5.0\/wiki\/Community-contribution [Accessed 11-11-2024]"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.1093\/database\/baac035","article-title":"A Simple Standard for Sharing Ontological Mappings (SSSOM)","volume":"2022","author":"Matentzoglu","journal-title":"Database (Oxford)"},{"key":"R22","unstructured":"Release notes v20240830. GitHub. URL: https:\/\/github.com\/OHDSI\/Vocabulary-v5.0\/releases\/tag\/v20240830_1725004358.000000 [Accessed 12-06-2025]"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2172-12-49","article-title":"Ontology-based Brucella vaccine literature indexing and systematic analysis of gene-vaccine association network","volume":"12","author":"Hur","journal-title":"BMC Immunol"},{"issue":"Suppl 2","key":"R24","doi-asserted-by":"publisher","DOI":"10.1186\/2041-1480-2-S2-S8","article-title":"Mining of vaccine-associated IFN-\u03b3 gene interaction networks using the Vaccine Ontology","volume":"2 Suppl 2","author":"Ozg\u00fcr","journal-title":"J Biomed Semantics"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.1186\/2041-1480-4-33","article-title":"Network-based analysis of vaccine-related associations reveals consistent knowledge with the Vaccine Ontology","volume":"4","author":"Zhang","journal-title":"J Biomed Semantics"},{"key":"R26","unstructured":"Lin AY Matheny M Williams A You SC . Medical device standard terminology overview, comparison and analysis. Observational Health Data Sciences and Informatics. 2024. URL: https:\/\/www.ohdsi.org\/2024showcase-30\/ [Accessed 30-08-2026]"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.2196\/56237","article-title":"Making metadata machine-readable as the first step to providing findable, accessible, interoperable, and reusable population health data: framework development and implementation study","volume":"16","author":"Amadi","journal-title":"Online J Public Health Inform"},{"key":"R28","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2024.1329630","article-title":"INSPIRE datahub: a pan-African integrated suite of services for harmonising longitudinal population health data using OHDSI tools","volume":"6","author":"Bhattacharjee","journal-title":"Front Digit Health"},{"key":"R29","unstructured":"Introducing the LOINC Ontology: a LOINC and SNOMED CT interoperability solution. SNOMED International. URL: https:\/\/loincsnomed.org\/ [Accessed 11-11-2024]"},{"issue":"e1","key":"R30","doi-asserted-by":"publisher","first-page":"e162","DOI":"10.1136\/amiajnl-2011-000583","article-title":"Portability of an algorithm to identify rheumatoid arthritis in electronic health records","volume":"19","author":"Carroll","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"R31","doi-asserted-by":"publisher","first-page":"1540","DOI":"10.1093\/jamia\/ocy101","article-title":"A case study evaluating the portability of an executable computable phenotype algorithm across multiple institutions and electronic health record environments","volume":"25","author":"Pacheco","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"R32","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1093\/jamia\/ocx138","article-title":"Clinical documentation variations and NLP system portability: a case study in asthma birth cohorts across institutions","volume":"25","author":"Sohn","journal-title":"J Am Med Inform Assoc"},{"key":"R33","unstructured":"Ostropolets A Chen R Spotnitz M Wang R Ryan P Hripcsak G . Phenotype algorithm and data source reporting in top clinical journals: where we are and where should we go. Observational Health Data Sciences and Informatics. 2020. URL: https:\/\/www.ohdsi.org\/2020-global-symposium-showcase-25\/ [Accessed 30-05-2026]"},{"issue":"4","key":"R34","doi-asserted-by":"publisher","first-page":"478","DOI":"10.5858\/arpa.2019-0055-OA","article-title":"A 20-year evaluation of LOINC in the United States\u2019 largest integrated health system","volume":"144","author":"Bhargava","journal-title":"Arch Pathol Lab Med"},{"key":"R35","volume":"2022","author":"Ostropolets","journal-title":"AMIA Annu Symp Proc"},{"key":"R36","volume":"2019","author":"Rasmussen","journal-title":"AMIA Annu Symp Proc"},{"issue":"6","key":"R37","doi-asserted-by":"publisher","first-page":"899","DOI":"10.1097\/TA.0000000000004050","article-title":"Defining the emergency general surgery patient population in the era of ICD-10: evaluating an established crosswalk from ICD-9 to ICD-10 diagnosis codes","volume":"95","author":"Dalton","journal-title":"J Trauma Acute Care Surg"},{"key":"R38","doi-asserted-by":"crossref","unstructured":"Philipp P Veloso J Appenzeller A Hartz T Beyerer J . Evaluation of an automated mapping from ICD-10 to SNOMED CT. In: 2022 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE; 2022. [doi: 10.1109\/CSCI58124.2022.00287]","DOI":"10.1109\/CSCI58124.2022.00287"},{"key":"R39","doi-asserted-by":"crossref","unstructured":"Rodrigues JM Robinson D Della Mea V et al. Semantic alignment between ICD-11 and SNOMED CT. In: Studies in Health Technology and Informatics. IOS Press; 2015:790-794. [doi: 10.3233\/978-1-61499-564-7-790]","DOI":"10.3233\/978-1-61499-564-7-790"},{"issue":"10","key":"R40","doi-asserted-by":"publisher","first-page":"1625","DOI":"10.1093\/jamia\/ocaa108","article-title":"A review of auditing techniques for the Unified Medical Language System","volume":"27","author":"Zheng","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"R41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3626960","article-title":"Enhancing human-in-the-loop ontology curation results through task design","volume":"16","author":"Tsaneva","journal-title":"ACM J Data Inf Qual"},{"issue":"2","key":"R42","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1093\/jamia\/ocaf184","article-title":"An exploratory analysis of SNOMED CT national editions","volume":"33","author":"Lee","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"R43","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1055\/a-2606-9411","article-title":"Semantic relations: extending SNOMED CT and Solor","volume":"16","author":"Resnick","journal-title":"Appl Clin Inform"},{"key":"R44","doi-asserted-by":"publisher","DOI":"10.2196\/69004","article-title":"Breaking digital health barriers through a large language model-based tool for automated observational medical outcomes partnership mapping: development and validation study","volume":"27","author":"Adams","journal-title":"J Med Internet Res"},{"key":"R45","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2025.103204","article-title":"LLM-based approaches for automated vocabulary mapping between SIGTAP and OMOP CDM concepts","volume":"168","author":"de Barros Vanzin","journal-title":"Artif Intell Med"},{"issue":"1","key":"R46","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-025-02669-0","article-title":"Semiautomatic mapping of a national drug terminology to standardised OMOP drug concepts using publicly available supplementary information","volume":"25","author":"Katsch","journal-title":"BMC Med Res Methodol"},{"issue":"4","key":"R47","doi-asserted-by":"publisher","first-page":"355","DOI":"10.4258\/hir.2024.30.4.355","article-title":"Mapping drug terms via integration of a retrieval-augmented generation algorithm with a large language model","volume":"30","author":"Kimura","journal-title":"Healthc Inform Res"},{"key":"R48","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2025.111418","article-title":"A dataset for mapping the Japanese drugs to RxNorm standard concepts","volume":"59","author":"Kimura","journal-title":"Data Brief"},{"key":"R49","unstructured":"Standardized vocabularies. Observational Health Data Sciences and Informatics (OHDSI). URL: https:\/\/athena.ohdsi.org\/vocabulary\/list [Accessed 02-09-2026]"},{"key":"R50","unstructured":"Network prevalence count. ATLAS. URL: https:\/\/atlas-demo.ohdsi.org\/#\/datasources [Accessed 02-09-2026]"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T19:44:04Z","timestamp":1789501444000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e92727"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,15]]},"references-count":50,"URL":"https:\/\/doi.org\/10.2196\/92727","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,15]]},"article-number":"v14i1e92727"}}