{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T14:25:53Z","timestamp":1788877553469,"version":"build-2803163510"},"reference-count":58,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Integrating data is essential for advancing clinical and epidemiological research. However, because datasets often describe variables (eg, demographic and health conditions) in diverse ways, the process of integrating and harmonizing variables from research studies remains a major bottleneck.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The objective was to assess a natural language processing\u2013based method to automate variable harmonization to achieve a scalable approach to integration of multiple datasets.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We developed a fully connected neural network (FCN) method, enhanced with contrastive learning, using domain-specific embeddings from the Bidirectional Encoder Representations from Transformers for Biomedical Text Mining language representation model, using 3 cardiovascular datasets: the Atherosclerosis Risk in Communities study, the Framingham Heart Study, and the Multi-Ethnic Study of Atherosclerosis. We used metadata variable descriptions and curated harmonized concepts as ground truth. We framed the problem as a paired sentence classification task. The accuracy of this method was compared with a logistic regression baseline method. To assess the generalizability of the trained models, we also evaluated their performance by separating the 3 datasets when preparing the training and validation sets.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The newly developed FCN achieved a top-5 accuracy of 98.95% (95% CI 98.31%\u201099.47%) and an area under the receiver operating characteristic (AUC) of 0.99 (95% CI 0.98\u20100.99), outperforming the standard logistic regression model, which exhibited a top-5 accuracy of 22.23% (95% CI 19.91%\u201024.87%) and an AUC of 0.82 (95% CI 0.81\u20100.83). The contrastive learning enhancement also outperformed the logistic regression model, although slightly below the base FCN model, exhibiting a top-5 accuracy of 89.88% (95% CI 87.88%\u201091.68%) and an AUC of 0.98 (95% CI 0.97\u20100.98).<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n This novel approach provides a scalable solution for harmonizing metadata across large-scale cohort studies. The proposed method significantly enhances the performance over the baseline method by using learned representations to categorize harmonized concepts more accurately for cohorts in cardiovascular disease and stroke.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/75608","type":"journal-article","created":{"date-parts":[[2025,6,14]],"date-time":"2025-06-14T13:30:10Z","timestamp":1749907810000},"page":"e75608-e75608","source":"Crossref","is-referenced-by-count":6,"title":["Automated Data Harmonization in Clinical Research: Natural Language Processing Approach"],"prefix":"10.2196","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5545-2256","authenticated-orcid":false,"given":"Pratheek","family":"Mallya","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4980-845X","authenticated-orcid":false,"given":"Ricardo","family":"Henao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7056-9559","authenticated-orcid":false,"given":"Chuan","family":"Hong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3431-2014","authenticated-orcid":false,"given":"Daniel","family":"Wojdyla","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6256-1907","authenticated-orcid":false,"given":"Tony","family":"Schibler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4700-9255","authenticated-orcid":false,"given":"Vihaan","family":"Manchanda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1968-2641","authenticated-orcid":false,"given":"Michael","family":"Pencina","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5103-5399","authenticated-orcid":false,"given":"Jennifer","family":"Hall","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1429-0662","authenticated-orcid":false,"given":"Juan","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,8,27]]},"reference":[{"issue":"8","key":"R1","doi-asserted-by":"publisher","first-page":"e93","DOI":"10.1161\/CIR.0000000000001123","article-title":"Heart disease and stroke statistics-2023 update: a report from the American Heart Association","volume":"147","author":"Tsao","journal-title":"Circulation"},{"issue":"4","key":"R2","doi-asserted-by":"publisher","first-page":"919","DOI":"10.21037\/cdt.2020.01.07","article-title":"Cardiovascular\/stroke risk predictive calculators: a comparison between statistical and machine learning models","volume":"10","author":"Jamthikar","journal-title":"Cardiovasc Diagn Ther"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-2265-0_2","article-title":"Integration of omics and phenotypic data for precision medicine","volume":"2486","author":"Zhao","journal-title":"Methods Mol Biol"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1111\/cts.12884","article-title":"Precision medicine, AI, and the future of personalized health care","volume":"14","author":"Johnson","journal-title":"Clinical Translational Sci"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.1016\/j.smhl.2021.100263","article-title":"A review of harmonization methods for studying dietary patterns","volume":"23","author":"Gurugubelli","journal-title":"Smart Health (2014)"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/52967","article-title":"Use of metadata-driven approaches for data harmonization in the medical domain: scoping review","volume":"12","author":"Peng","journal-title":"JMIR Med Inform"},{"issue":"11","key":"R7","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1161\/CIRCOUTCOMES.123.009938","article-title":"Facilitating harmonization of variables in Framingham, MESA, ARIC, and REGARDS studies through a metadata repository","volume":"16","author":"Mallya","journal-title":"Circ: Cardiovascular Quality and Outcomes"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-024-02956-3","article-title":"A general primer for data harmonization","volume":"11","author":"Cheng","journal-title":"Sci Data"},{"issue":"12","key":"R9","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1093\/aje\/kwad153","article-title":"Large-scale data harmonization across prospective studies","volume":"192","author":"Pan","journal-title":"Am J Epidemiol"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.23889\/ijpds.v6i1.1680","article-title":"Data harmonization and data pooling from cohort studies: a practical approach for data management","volume":"6","author":"Adhikari","journal-title":"Int J Popul Data Sci"},{"key":"R11","doi-asserted-by":"crossref","unstructured":"Sony P . Concept-based electronic health record retrieval system in healthcare IOT. In: Cognitive Informatics and Soft Computing. Vol 768. Springer; 2019:175-188. [doi: 10.1007\/978-981-13-0617-4_17]","DOI":"10.1007\/978-981-13-0617-4_17"},{"issue":"Suppl 1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-8-S1-S6","article-title":"Automatic medical encoding with SNOMED categories","volume":"8 Suppl 1","author":"Ruch","journal-title":"BMC Med Inform Decis Mak"},{"issue":"4","key":"R13","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1373\/49.4.624","article-title":"LOINC, a universal standard for identifying laboratory observations: a 5-year update","volume":"49","author":"McDonald","journal-title":"Clin Chem"},{"key":"R14","unstructured":"W. H. Organization and others, International classification of diseases. Basic Tabulation List with Alphabetic Index. World Health Organization; 1978."},{"key":"R15","unstructured":"W. H. Organization, International Statistical Classification of Diseases and related health problems. Alphabetical Index. Vol 3. World Health Organization; 2004."},{"key":"R16","unstructured":"Abraham M Ahlman JT Boudreau AJ et al. CPT 2011 Standard Edition. 4th ed. American Medical Association; 2010. ISBN: 1603592164, 9781603592161"},{"key":"R17","unstructured":"Clinical Classifications Software (CCS) for ICD-9-CM. URL: https:\/\/www.oit.va.gov\/Services\/TRM\/ToolPage.aspx?tid=7602 [Accessed 09-12-2024]"},{"issue":"4","key":"R18","doi-asserted-by":"publisher","first-page":"634","DOI":"10.1016\/j.jbi.2012.02.011","article-title":"Utilizing RxNorm to support practical computing applications: capturing medication history in live electronic health records","volume":"45","author":"Bennett","journal-title":"J Biomed Inform"},{"key":"R19","unstructured":"National Drug Code Directory. U.S. Food and Drug Association. URL: https:\/\/www.fda.gov\/drugs\/drug-approvals-and-databases\/national-drug-code-directory [Accessed 09-12-2024]"},{"key":"R20","unstructured":"Devlin J Chang MW Lee K Toutanova K . BERT: pre-training of deep bidirectional transformers for language understanding. arXiv. Preprint posted online on May 24, 2019. URL: http:\/\/arxiv.org\/abs\/1810.04805 [Accessed 18-08-2025] [doi: 10.48550\/arXiv.1810.04805]"},{"issue":"4","key":"R21","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","journal-title":"Bioinformatics"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-021-00519-z","article-title":"Clinical knowledge extraction via sparse embedding regression (KESER) with multi-center large scale electronic health record data","volume":"4","author":"Hong","journal-title":"NPJ Digit Med"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Kartchner D Christensen T Humpherys J Wade S . Code2Vec: embedding and clustering medical diagnosis data. Presented at: 2017 IEEE International Conference on Healthcare Informatics (ICHI); Aug 23-26, 2017:386-390; Park City, UT, USA. [doi: 10.1109\/ICHI.2017.94]","DOI":"10.1109\/ICHI.2017.94"},{"key":"R24","doi-asserted-by":"crossref","unstructured":"Choi E Bahadori MT Searles E et al. Multi-layer representation learning for medical concepts. Presented at: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Aug 13, 2016:ACM. 1495-1504; San Francisco California USA. [doi: 10.1145\/2939672.2939823]","DOI":"10.1145\/2939672.2939823"},{"key":"R25","unstructured":"H\u00e9naff OJ et al. Data-efficient image recognition with contrastive predictive coding. Preprint posted online on 2019. [doi: 10.48550\/ARXIV.1905.09272]"},{"issue":"4","key":"R26","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1001\/jama.2022.24683","article-title":"Predictive accuracy of stroke risk prediction models across Black and White race, sex, and age groups","volume":"329","author":"Hong","journal-title":"JAMA"},{"issue":"14","key":"R27","doi-asserted-by":"publisher","first-page":"i37","DOI":"10.1093\/bioinformatics\/btx228","article-title":"Deep learning with word embeddings improves biomedical named entity recognition","volume":"33","author":"Habibi","journal-title":"Bioinformatics"},{"key":"R28","doi-asserted-by":"crossref","unstructured":"Lin C Miller T Dligach D Bethard S Savova G . A BERT-based universal model for both within-and cross-sentence clinical temporal relation extraction. Presented at: Proceedings of the 2nd Clinical Natural Language Processing Workshop; Jun 2019:65-71; Minneapolis, Minnesota, USA. [doi: 10.18653\/v1\/W19-1908]","DOI":"10.18653\/v1\/W19-1908"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Wiese G Weissenborn D Neves M . Neural domain adaptation for biomedical question answering. Presented at: Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017); 2017; Vancouver, Canada. [doi: 10.18653\/v1\/K17-1029]","DOI":"10.18653\/v1\/K17-1029"},{"key":"R30","doi-asserted-by":"crossref","unstructured":"Sutton A Cristianini N . On the learnability of concepts: with applications to comparing word embedding algorithms. In: Maglogiannis I Iliadis L Pimenidis E , editors. Artificial Intelligence Applications and Innovations. Vol 584. Springer International Publishing; 2020:420-432. [doi: 10.1007\/978-3-030-49186-4_35]","DOI":"10.1007\/978-3-030-49186-4_35"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Mueller J Thyagarajan A . Siamese recurrent architectures for learning sentence similarity. Presented at: Proceedings of the AAAI conference on artificial intelligence; 2016. [doi: 10.1609\/aaai.v30i1.10350]","DOI":"10.1609\/aaai.v30i1.10350"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Corley C Mihalcea R . Measuring the semantic similarity of texts. In: Dolan B Dagan I , editors. Presented at: Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment; Jun 18, 2005; Ann Arbor, Michigan. [doi: 10.3115\/1631862.1631865]","DOI":"10.3115\/1631862.1631865"},{"key":"R33","unstructured":"Nair V Hinton GE . Rectified linear units improve restricted boltzmann machines. Presented at: Proceedings of the 27th International Conference on Machine Learning (ICML-10); 2010:807-814; Madison, Wisconsin, USA. URL: https:\/\/dl.acm.org\/doi\/10.5555\/3104322.3104425 [Accessed 18-08-2025]"},{"key":"R34","volume":"32","author":"Zhang","journal-title":"Adv Neural Inf Process Syst"},{"key":"R35","unstructured":"Kingma DP Ba J . Adam: a method for stochastic optimization. arXiv. Preprint posted online on 2014."},{"key":"R36","unstructured":"Chen T Kornblith S Norouzi M Hinton G . A simple framework for contrastive learning of visual representations. arXiv. Preprint posted online on Feb 13, 2020. URL: http:\/\/arxiv.org\/abs\/2002.05709 [Accessed 18-08-2025] [doi: 10.48550\/arXiv.2002.05709]"},{"key":"R37","unstructured":"Li Y Vinyals O . Representation learning with contrastive predictive coding. Preprint posted online on 2018. [doi: 10.48550\/ARXIV.1807.03748]"},{"issue":"7","key":"R38","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","article-title":"The use of the area under the ROC curve in the evaluation of machine learning algorithms","volume":"30","author":"Bradley","journal-title":"Pattern Recognit DAGM"},{"issue":"1","key":"R39","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a receiver operating characteristic (ROC) curve","volume":"143","author":"Hanley","journal-title":"Radiology"},{"key":"R40","volume":"12","author":"Pedregosa","journal-title":"J Mach Learn Res"},{"key":"R41","unstructured":"Krizhevsky A Sutskever I Hinton GE . Imagenet classification with deep convolutional neural networks. In: Adv Neural Inf Process Syst. Vol 25. 2012."},{"key":"R42","doi-asserted-by":"crossref","unstructured":"Efron B Tibshirani RJ . An introduction to the bootstrap. In: Chapman and Hall\/CRC. 1994. [doi: 10.1201\/9780429246593]","DOI":"10.1201\/9780429246593"},{"issue":"4","key":"R43","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1038\/ejhg.2015.165","article-title":"Harmonising and linking biomedical and clinical data across disparate data archives to enable integrative cross-biobank research","volume":"24","author":"Spjuth","journal-title":"Eur J Hum Genet"},{"issue":"7","key":"R44","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0328262","article-title":"A natural language processing approach to support biomedical data harmonization: leveraging large language models","volume":"20","author":"Li","journal-title":"PLoS ONE"},{"key":"R45","doi-asserted-by":"publisher","DOI":"10.2196\/54133","article-title":"Robust automated harmonization of heterogeneous data through ensemble machine learning: algorithm development and validation study","volume":"13","author":"Yang","journal-title":"JMIR Med Inform"},{"issue":"1","key":"R46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","journal-title":"ACM Trans Comput Healthcare"},{"key":"R47","doi-asserted-by":"crossref","unstructured":"Peng Y Yan S Lu Z . Transfer learning in biomedical natural language processing: an evaluation of BERT and elmo on ten benchmarking datasets. Presented at: Proceedings of the 18th BioNLP Workshop and Shared Task. 2019:Association for Computational Linguistics. 58-65; Florence, Italy. [doi: 10.18653\/v1\/W19-5006]","DOI":"10.18653\/v1\/W19-5006"},{"issue":"7","key":"R48","doi-asserted-by":"publisher","first-page":"1172","DOI":"10.1093\/jamia\/ocac054","article-title":"Harmonizing units and values of quantitative data elements in a very large nationally pooled electronic health record (EHR) dataset","volume":"29","author":"Bradwell","journal-title":"J Am Med Inform Assoc"},{"issue":"7972","key":"R49","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","journal-title":"Nature New Biol"},{"issue":"13s","key":"R50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3582688","article-title":"A comprehensive survey of few-shot learning: evolution, applications, challenges, and opportunities","volume":"55","author":"Song","journal-title":"ACM Comput Surv"},{"issue":"1","key":"R51","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/JPROC.2020.3004555","article-title":"A comprehensive survey on transfer learning","volume":"109","author":"Zhuang","journal-title":"Proc IEEE"},{"issue":"3","key":"R52","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btad103","article-title":"On the effectiveness of compact biomedical transformers","volume":"39","author":"Rohanian","journal-title":"Bioinformatics"},{"key":"R53","doi-asserted-by":"crossref","unstructured":"Graves A Mohamed A r Hinton G Mohamed A r . Speech recognition with deep recurrent neural networks. Presented at: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing; May 26-31, 2013:6645-6649; Vancouver, BC, Canada. [doi: 10.1109\/ICASSP.2013.6638947]","DOI":"10.1109\/ICASSP.2013.6638947"},{"issue":"8","key":"R54","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","journal-title":"Neural Comput"},{"key":"R55","unstructured":"Brown TB Mann B Ryder N et al. Language models are few-shot learners. arXiv. Preprint posted online on 2020. [doi: 10.48550\/ARXIV.2005.14165]"},{"key":"R56","author":"Chowdhery","journal-title":"J Mach Learn Res"},{"key":"R57","unstructured":"Touvron H Lavril T Izacard G et al. LLaMA: open and efficient foundation language models. arXiv. Preprint posted online on Feb 27, 2023. [doi: 10.48550\/ARXIV.2302.13971]"},{"key":"R58","doi-asserted-by":"crossref","unstructured":"Kanakarajan K raj Kundumani B Abraham A Sankarasubbu M . BioSimCSE: biomedical sentence embeddings using contrastive learning. Presented at: Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI); 2022:81-86; Abu Dhabi, United Arab Emirates (Hybrid. [doi: 10.18653\/v1\/2022.louhi-1.10]","DOI":"10.18653\/v1\/2022.louhi-1.10"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2025\/1\/e75608","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:06:41Z","timestamp":1782842801000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2025\/1\/e75608"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,27]]},"references-count":58,"URL":"https:\/\/doi.org\/10.2196\/75608","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,27]]},"article-number":"v9i8e75608"}}