{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T03:02:52Z","timestamp":1787022172176,"version":"build-2736575974"},"reference-count":27,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Accurate identification of graft loss in Electronic Medical Records of kidney transplant recipients is essential but challenging due to inconsistent and not mandatory International Classification of Diseases (ICD) codes. We developed and validated Natural Language Processing (NLP) and machine learning models to classify the status of kidney allografts in unstructured text in EMRs written in Spanish.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a retrospective cohort of 2712 patients transplanted between July 2008 and January 2023, analyzing 117,566 unstructured medical records. NLP involved text normalization, tokenization, stopwords removal, spell-checking, elimination of low-frequency words and stemming. Data was split in training, validation and test sets. Data balance was performed using undersampling technique. Feature selection was performed using LASSO regression. We developed, validated and tested Logistic Regression, Random Forest, and Neural Networks models using 10-fold cross-validation. Performance metrics included area under the curve, F1 Score, accuracy, sensitivity, specificity, Negative Predictive Value, and Positive Predictive Value.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The test performance results showed that the Random Forest model achieved the highest AUC (0.98) and F1 score (0.65). However, it had a modest sensitivity (0.76) and a relatively low PPV (0.56), implying a significant number of false positives. The Neural Network model also performed well with a high AUC (0.98) and reasonable F1 score (0.61), but its PPV (0.49) was lower, indicating more false positives. The Logistic Regression model, while having the lowest AUC (0.91) and F1 score (0.49), showed the highest sensitivity (0.83) with the lowest PPV (0.35).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec004\">\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>We developed and validated three machine learning models combined with NLP techniques for unstructured texts written in Spanish. The models performed well on the validation set but showed modest performance on the test set due to data imbalance. These models could be adapted for clinical practice, though they may require additional manual work due to high false positive rates.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0322587","type":"journal-article","created":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T13:28:03Z","timestamp":1746710883000},"page":"e0322587","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["Using Natural Language Processing and Machine Learning to classify the status of kidney allograft in Electronic Medical Records written in Spanish"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3940-1413","authenticated-orcid":true,"given":"Andrea","family":"Garcia-Lopez","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8888-9411","authenticated-orcid":true,"given":"Juliana","family":"Cuervo-Rojas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan","family":"Garcia-Lopez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6307-0848","authenticated-orcid":true,"given":"Fernando","family":"Giron-Luque","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,5,8]]},"reference":[{"issue":"2","key":"pone.0322587.ref001","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.kint.2019.12.001","article-title":"Considerations for advancing nephrology research and practice through natural language processing","volume":"97","author":"SK Parr","year":"2020","journal-title":"Kidney Int"},{"issue":"10","key":"pone.0322587.ref002","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1002\/pds.3856","article-title":"Identifying health outcomes in healthcare databases","volume":"24","author":"S Lanes","year":"2015","journal-title":"Pharmacoepidemiol Drug Saf"},{"issue":"1","key":"pone.0322587.ref003","doi-asserted-by":"crossref","first-page":"1\u20139","DOI":"10.1186\/s12911-022-02017-y","article-title":"Natural language processing for identification of hypertrophic cardiomyopathy patients from cardiac magnetic resonance reports","volume":"22","author":"N Dewaswala","year":"2022","journal-title":"BMC Med Inform Decis Mak"},{"issue":"2","key":"pone.0322587.ref004","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/s10654-018-0470-0","article-title":"Use of natural language processing in electronic medical records to identify pregnant women with suicidal behavior: towards a solution to the complex classification problem","volume":"34","author":"Q-Y Zhong","year":"2019","journal-title":"Eur J Epidemiol"},{"key":"pone.0322587.ref005","doi-asserted-by":"crossref","first-page":"119171","DOI":"10.1016\/j.eswa.2022.119171","article-title":"Classification of neurologic outcomes from medical notes using natural language processing","volume":"214","author":"MB Fernandes","year":"2023","journal-title":"Expert Syst Appl"},{"issue":"3","key":"pone.0322587.ref006","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.cvdhj.2021.03.003","article-title":"Large-scale identification of aortic stenosis and its severity using natural language processing on electronic health records","volume":"2","author":"MD Solomon","year":"2021","journal-title":"Cardiovasc Digit Health J"},{"issue":"4","key":"pone.0322587.ref007","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1111\/ajt.15705","article-title":"Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation","volume":"20","author":"AM Placona","year":"2020","journal-title":"Am J Transplant"},{"issue":"1","key":"pone.0322587.ref008","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1001\/jamasurg.2023.4679","article-title":"Prediction of High-Risk Donors for Kidney Discard and Nonrecovery Using Structured Donor Characteristics and Unstructured Donor Narratives","volume":"159","author":"J Sageshima","year":"2024","journal-title":"JAMA Surg"},{"issue":"3","key":"pone.0322587.ref009","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1111\/ajt.14099","article-title":"Big Data, Predictive Analytics, and Quality Improvement in Kidney Transplantation: A Proof of Concept","volume":"17","author":"TR Srinivas","year":"2017","journal-title":"Am J Transplant"},{"issue":"11","key":"pone.0322587.ref010","doi-asserted-by":"crossref","DOI":"10.1007\/s10462-024-10921-0","article-title":"Large language models in medical and healthcare fields: applications, advances, and challenges","volume":"57","author":"D Wang","year":"2024","journal-title":"Artif Intell Rev"},{"key":"pone.0322587.ref011","first-page":"2024.08.11.24311828","article-title":"Generative Large Language Models in Electronic Health Records for Patient Care Since 2023: A Systematic Review","author":"X Du","year":"2024","journal-title":"medRxiv"},{"key":"pone.0322587.ref012","volume-title":"Transplant_words_NLP [Internet]. Kaggle","author":"A Garcia-Lopez","year":"2024"},{"issue":"1","key":"pone.0322587.ref013","first-page":"82","article-title":"ROSE: a Package for Binary Imbalanced Learning","volume":"6","author":"N Lunardon","year":"2014","journal-title":"R Journal"},{"issue":"12","key":"pone.0322587.ref014","first-page":"2222","article-title":"What is Machine Learning? A Primer for the Epidemiologist","volume":"188","author":"Q Bi","year":"2019","journal-title":"Am J Epidemiol"},{"issue":"3","key":"pone.0322587.ref015","doi-asserted-by":"crossref","first-page":"128","DOI":"10.14445\/22312803\/IJCTT-V48P126","article-title":"Supervised Machine Learning Algorithms: Classification and Comparison","volume":"48","author":"FY Osisanwo","year":"2017","journal-title":"International Journal of Computer Trends and Technology"},{"issue":"1","key":"pone.0322587.ref016","article-title":"Cerebrovascular disease case identification in inpatient electronic medical record data using natural language processing","volume":"10","author":"J Pan","year":"2023","journal-title":"Brain Inform"},{"issue":"1","key":"pone.0322587.ref017","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1186\/s12874-024-02231-4","article-title":"Identification of patients\u2019 smoking status using an explainable AI approach: a Danish electronic health records case study","volume":"24","author":"A Ebrahimi","year":"2024","journal-title":"BMC Med Res Methodol"},{"issue":"3","key":"pone.0322587.ref018","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.ekir.2022.12.006","article-title":"Predicting Kidney Transplant Recipient Cohorts\u2019 30-Day Rehospitalization Using Clinical Notes and Electronic Health Care Record Data","volume":"8","author":"M Arenson","year":"2023","journal-title":"Kidney Int Rep"},{"issue":"1","key":"pone.0322587.ref019","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/s13326-018-0179-8","article-title":"Clinical Natural Language Processing in languages other than English: opportunities and challenges","volume":"9","author":"A N\u00e9v\u00e9ol","year":"2018","journal-title":"J Biomed Semantics"},{"issue":"1","key":"pone.0322587.ref020","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1055\/s-0043-1768726","article-title":"Exploring the Latest Highlights in Medical Natural Language Processing across Multiple Languages: A Survey","volume":"32","author":"A Shaitarova","year":"2023","journal-title":"Yearb Med Inform"},{"issue":"3","key":"pone.0322587.ref021","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1093\/ehjdh\/ztae008","article-title":"Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease","volume":"5","author":"M Falter","year":"2024","journal-title":"Eur Heart J Digit Health"},{"key":"pone.0322587.ref022","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.ajo.2024.01.030","article-title":"Using Natural Language Processing to Identify Different Lens Pathology in Electronic Health Records","volume":"262","author":"JD Stein","year":"2024","journal-title":"Am J Ophthalmol"},{"issue":"10","key":"pone.0322587.ref023","doi-asserted-by":"crossref","first-page":"e30106","DOI":"10.1016\/j.heliyon.2024.e30106","article-title":"Automated vs. manual coding of neuroimaging reports via natural language processing, using the international classification of diseases, tenth revision","volume":"10","author":"AM McKinney","year":"2024","journal-title":"Heliyon"},{"key":"pone.0322587.ref024","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.thromres.2021.04.020","article-title":"A comparison of natural language processing to ICD-10 codes for identification and characterization of pulmonary embolism","volume":"203","author":"SA Johnson","year":"2021","journal-title":"Thromb Res"},{"issue":"1","key":"pone.0322587.ref025","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1007\/s10916-024-02045-3","article-title":"The Breakthrough of Large Language Models Release for Medical Applications: 1-Year Timeline and Perspectives","volume":"48","author":"M Cascella","year":"2024","journal-title":"J Med Syst"},{"key":"pone.0322587.ref026","doi-asserted-by":"crossref","first-page":"e56955","DOI":"10.2196\/56955","article-title":"Disambiguating Clinical Abbreviations by One-to-All Classification: Algorithm Development and Validation Study","volume":"12","author":"S-F Sung","year":"2024","journal-title":"JMIR Med Inform"},{"issue":"11","key":"pone.0322587.ref027","doi-asserted-by":"crossref","first-page":"3041","DOI":"10.3390\/jcm13113041","article-title":"Clinical and Surgical Applications of Large Language Models: A Systematic Review","volume":"13","author":"SM Pressman","year":"2024","journal-title":"J Clin Med"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0322587","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T13:28:13Z","timestamp":1746710893000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0322587"}},"subtitle":[],"editor":[{"given":"Abdulaziz T.","family":"Bako","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,5,8]]},"references-count":27,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0322587","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,8]]}}}