{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T16:43:58Z","timestamp":1771951438988,"version":"3.50.1"},"reference-count":102,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T00:00:00Z","timestamp":1738800000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Austrian Ministry of Transport, Innovation and Technology (BMVIT)"},{"name":"BMK and BMAW"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Electronic health record (EHR) de-identification is crucial for publishing or sharing medical data without violating the patient\u2019s privacy. Protected health information (PHI) is abundant in EHRs, and privacy regulations worldwide mandate de-identification before downstream tasks are performed. The ever-growing data generation in healthcare and the advent of generative artificial intelligence have increased the demand for de-identified EHRs and highlighted privacy issues with large language models (LLMs), especially data transmission to cloud-based LLMs. In this study, we benchmark ten LLMs for de-identifying EHRs in English and German. We then compare de-identification performance for in-context learning and full model fine-tuning and analyze the limitations of LLMs for this task. Our experimental evaluation shows that LLMs effectively de-identify EHRs in both languages. Moreover, in-context learning with a one-shot setting boosts de-identification performance without the costly full fine-tuning of the LLMs.<\/jats:p>","DOI":"10.3390\/info16020112","type":"journal-article","created":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T06:30:29Z","timestamp":1738823429000},"page":"112","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Large Language Models for Electronic Health Record De-Identification in English and German"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7196-9095","authenticated-orcid":false,"given":"Samuel","family":"Sousa","sequence":"first","affiliation":[{"name":"Know Center Research GmbH, 8010 Graz, Austria"},{"name":"Institute of Human-Centred Computing, Graz University of Technology, 8010 Graz, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-4779-0762","authenticated-orcid":false,"given":"Michael","family":"Jantscher","sequence":"additional","affiliation":[{"name":"Know Center Research GmbH, 8010 Graz, Austria"},{"name":"Institute of Human-Centred Computing, Graz University of Technology, 8010 Graz, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8398-5292","authenticated-orcid":false,"given":"Mark","family":"Kr\u00f6ll","sequence":"additional","affiliation":[{"name":"Know Center Research GmbH, 8010 Graz, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0202-6100","authenticated-orcid":false,"given":"Roman","family":"Kern","sequence":"additional","affiliation":[{"name":"Know Center Research GmbH, 8010 Graz, Austria"},{"name":"Institute of Human-Centred Computing, Graz University of Technology, 8010 Graz, Austria"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"S34","DOI":"10.1016\/j.jbi.2017.05.023","article-title":"De-identification of clinical notes via recurrent neural network and conditional random field","volume":"75","author":"Liu","year":"2017","journal-title":"J. Biomed. Inform."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-020-00351-4","article-title":"Survey on RNN and CRF models for de-identification of medical free text","volume":"7","author":"Leevy","year":"2020","journal-title":"J. Big Data"},{"key":"ref_3","unstructured":"Act (1996). Health insurance portability and accountability act of 1996. Public Law, 104, 191."},{"key":"ref_4","unstructured":"European Commission (2024, December 30). A New Era for Data Protection in the EU. Available online: https:\/\/commission.europa.eu\/document\/download\/7fa5e36d-6412-4b44-9a2d-12d4838fd4c6_en?filename=data-protection-factsheet-changes_en.pdf."},{"key":"ref_5","unstructured":"Liu, Z., Huang, Y., Yu, X., Zhang, L., Wu, Z., Cao, C., Dai, H., Zhao, L., Li, Y., and Shu, P. (2023). Deid-gpt: Zero-shot medical text de-identification by gpt-4. arXiv."},{"key":"ref_6","unstructured":"Patil, H.K., and Seshadri, R. (July, January 27). Big data security and privacy issues in healthcare. Proceedings of the 2014 IEEE International Congress on Big Data, Anchorage, AK, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1184","DOI":"10.1016\/j.ijinfomgt.2016.08.002","article-title":"Re-identification attacks\u2014A systematic literature review","volume":"36","author":"Jeary","year":"2016","journal-title":"Int. J. Inf. Manag."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, P., and Kamel Boulos, M.N. (2023). Generative AI in medicine and healthcare: Promises, opportunities and challenges. Future Internet, 15.","DOI":"10.3390\/fi15090286"},{"key":"ref_9","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 6000\u20136010."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s10916-024-02043-5","article-title":"Transformer Models in Healthcare: A Survey and Thematic Analysis of Potentials, Shortcomings and Risks","volume":"48","author":"Denecke","year":"2024","journal-title":"J. Med. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Van Veen, D., Van Uden, C., Blankemeier, L., Delbrouck, J.B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E.P., and Seehofnerova, A. (2023). Clinical text summarization: Adapting large language models can outperform human experts. Res. Sq.","DOI":"10.21203\/rs.3.rs-3483777\/v1"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chintagunta, B., Katariya, N., Amatriain, X., and Kannan, A. (2021, January 6\u20137). Medically aware GPT-3 as a data generator for medical dialogue summarization. Proceedings of the Machine Learning for Healthcare Conference, PMLR, Virtual Event.","DOI":"10.18653\/v1\/2021.nlpmc-1.9"},{"key":"ref_13","unstructured":"Xu, B., Gil-Jardin\u00e9, C., Thiessard, F., Tellier, E., Avalos, M., and Lagarde, E. (2020, January 17\u201320). Pre-training a neural language model improves the sample efficiency of an emergency room classification model. Proceedings of the FLAIRS-33-Thirty-Third International Flairs Conference, North Miami Beach, FL, USA."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1007\/s10462-022-10204-6","article-title":"How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing","volume":"56","author":"Sousa","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_15","unstructured":"Trienes, J., Trieschnigg, D., Seifert, C., and Hiemstra, D. (2020). Comparing rule-based, feature-based and deep neural methods for de-identification of dutch medical records. arXiv."},{"key":"ref_16","unstructured":"Kolditz, T., Lohr, C., Hellrich, J., Modersohn, L., Betz, B., Kiehntopf, M., and Hahn, U. (2019). Annotating German clinical documents for de-identification. MEDINFO 2019: Health and Wellbeing e-Networks for All, IOS Press BV."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Rehm, G., and Uszkoreit, H. (2012). The German Language in the European Information Society. The German Language in the Digital Age, Springer.","DOI":"10.1007\/978-3-642-27166-3"},{"key":"ref_18","unstructured":"Borchert, F., Lohr, C., Modersohn, L., Witt, J., Langer, T., Follmann, M., Gietzelt, M., Arnrich, B., Hahn, U., and Schapranow, M.P. (2022, January 20\u201325). GGPONC 2.0\u2014The German clinical guideline corpus for oncology: Curation workflow, annotation policy, baseline NER taggers. Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marseille, France."},{"key":"ref_19","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_20","unstructured":"OpenAI (2024, November 12). GPT-4 Technical Report. Available online: https:\/\/openai.com\/research\/gpt-4."},{"key":"ref_21","unstructured":"AI@Meta (2025, January 27). Llama 3 Model Card. Available online: https:\/\/github.com\/meta-llama\/llama3\/blob\/main\/MODEL_CARD.md."},{"key":"ref_22","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2288-10-70","article-title":"Automatic de-identification of textual documents in the electronic health record: A review of recent research","volume":"10","author":"Meystre","year":"2010","journal-title":"BMC Med. Res. Methodol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"680","DOI":"10.5858\/2003-127-680-CMDS","article-title":"Concept-match medical data scrubbing: How pathology text can be used in research","volume":"127","author":"Berman","year":"2003","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1472-6947-6-12","article-title":"Development and evaluation of an open source software tool for deidentification of pathology reports","volume":"6","author":"Beckwith","year":"2006","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1197\/jamia.M2702","article-title":"A software tool for removing patient identifying information from clinical documents","volume":"15","author":"Friedlin","year":"2008","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.artmed.2007.10.001","article-title":"A de-identifier for medical discharge summaries","volume":"42","author":"Uzuner","year":"2008","journal-title":"Artif. Intell. Med."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-Vector Networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1197\/jamia.M2435","article-title":"Rapidly retargetable approaches to de-identification in medical records","volume":"14","author":"Wellner","year":"2007","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_30","unstructured":"Lafferty, J., McCallum, A., and Pereira, F. (July, January 28). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the Icml, Williamstown, MA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1016\/j.ijmedinf.2010.09.007","article-title":"The MITRE Identification Scrubber Toolkit: Design, training, and assessment","volume":"79","author":"Aberdeen","year":"2010","journal-title":"Int. J. Med. Inform."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"S30","DOI":"10.1016\/j.jbi.2015.06.015","article-title":"Automatic detection of protected health information from clinic narratives","volume":"58","author":"Yang","year":"2015","journal-title":"J. Biomed. Inform."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"S11","DOI":"10.1016\/j.jbi.2015.06.007","article-title":"Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2\/UTHealth shared task Track 1","volume":"58","author":"Stubbs","year":"2015","journal-title":"J. Biomed. Inform."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TNNLS.2021.3084827","article-title":"A survey of convolutional neural networks: Analysis, applications, and prospects","volume":"33","author":"Li","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1038\/s41586-020-2669-y","article-title":"Illuminating the dark spaces of healthcare with ambient intelligence","volume":"585","author":"Haque","year":"2020","journal-title":"Nature"},{"key":"ref_36","first-page":"1463","article-title":"An overview of healthcare data analytics with applications to the COVID-19 pandemic","volume":"8","author":"Fei","year":"2021","journal-title":"IEEE Trans. Big Data"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hang, C.N., Tsai, Y.Z., Yu, P.D., Chen, J., and Tan, C.W. (2023). Privacy-enhancing digital contact tracing with machine learning for pandemic response: A comprehensive review. Big Data Cogn. Comput., 7.","DOI":"10.3390\/bdcc7020108"},{"key":"ref_38","unstructured":"Chen, H., Lin, Z., Ding, G., Lou, J., Zhang, Y., and Karlsson, B. (February, January 27). GRN: Gated relation network to enhance convolutional neural network for named entity recognition. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s11071-021-07160-1","article-title":"Estimating the state of epidemics spreading with graph neural networks","volume":"109","author":"Tomy","year":"2022","journal-title":"Nonlinear Dyn."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Tan, C.W., Yu, P.D., Chen, S., and Poor, H.V. (2022). Deeptrace: Learning to optimize contact tracing in epidemic networks with graph neural networks. arXiv.","DOI":"10.21203\/rs.3.rs-2461064\/v1"},{"key":"ref_41","first-page":"283","article-title":"Impact of de-identification on clinical text classification using traditional and deep learning classifiers","volume":"264","author":"Obeid","year":"2019","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014, January 25\u201329). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Netw."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1093\/jamia\/ocw156","article-title":"De-identification of patient notes with recurrent neural networks","volume":"24","author":"Dernoncourt","year":"2017","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Ahmed, T., Aziz, M.M.A., and Mohammed, N. (2020). De-identification of electronic health record using neural network. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-75544-1"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Cho, K., van Merri\u00ebnboer, B., G\u00fcl\u00e7ehre, \u00c7., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25\u201329). Learning Phrase Representations using RNN Encoder\u2013Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_48","unstructured":"Richter-Pechanski, P., Amr, A., Katus, H.A., and Dieterich, C. (2019, January 8\u201311). Deep Learning Approaches Outperform Conventional Strategies in De-Identification of German Medical Reports. Proceedings of the GMDS, Dortmund, Germany."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Baumgartner, M., Schreier, G., Hayn, D., Kreiner, K., Haider, L., Wiesm\u00fcller, F., Brunelli, L., and P\u00f6lzl, G. (2022). Impact analysis of De-identification in clinical notes classification. dHealth 2022, IOS Press BV.","DOI":"10.3233\/SHTI220368"},{"key":"ref_50","unstructured":"Eder, E., Krieg-Holz, U., and Hahn, U. (2020, January 11\u201316). CodE Alltag 2.0\u2014A pseudonymized German-language email corpus. Proceedings of the Twelfth Language Resources and Evaluation Conference, Marseille, France."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Kocaman, V., Mellah, Y., Haq, H., and Talby, D. (2023, January 7). Automated de-identification of arabic medical records. Proceedings of the ArabicNLP 2023, Singapore.","DOI":"10.18653\/v1\/2023.arabicnlp-1.4"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1186\/s12911-018-0598-6","article-title":"Leveraging text skeleton for de-identification of electronic medical records","volume":"18","author":"Zhao","year":"2018","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1016\/j.tele.2017.08.002","article-title":"DEDUCE: A pattern matching method for automatic de-identification of Dutch medical text","volume":"35","author":"Menger","year":"2018","journal-title":"Telemat. Inform."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Bourdois, L., Avalos, M., Chenais, G., Thiessard, F., Revel, P., Gil-Jardin\u00e9, C., and Lagarde, E. (2021). De-identification of emergency medical records in French: Survey and comparison of state-of-the-art automated systems. Int. Flairs Conf. Proc., 34.","DOI":"10.32473\/flairs.v34i1.128480"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"19097","DOI":"10.1109\/ACCESS.2021.3054479","article-title":"A novel covid-19 data set and an effective deep learning approach for the de-identification of italian medical records","volume":"9","author":"Catelli","year":"2021","journal-title":"IEEE Access"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13326-020-00227-9","article-title":"De-identifying free text of Japanese electronic health records","volume":"11","author":"Kajiyama","year":"2020","journal-title":"J. Biomed. Semant."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"7","DOI":"10.3346\/jkms.2015.30.1.7","article-title":"A de-identification method for bilingual clinical texts of various note types","volume":"30","author":"Shin","year":"2015","journal-title":"J. Korean Med. Sci."},{"key":"ref_58","unstructured":"Br\u00e5then, S., Wie, W., and Dalianis, H. (June, January 31). Creating and evaluating a synthetic Norwegian clinical corpus for de-identification. Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), Reykjavik, Iceland (Online)."},{"key":"ref_59","unstructured":"Prado, C.B., Gumiel, Y.B., Schneider, E.T.R., Cintho, L.M.M., de Souza, J.V.A., Oliveira, L.E.S.e., Paraiso, E.C., Rebelo, M.S., Gutierrez, M.A., and Pires, F.A. (2022, January 24\u201328). De-Identification Challenges in Real-World Portuguese Clinical Texts. Proceedings of the Latin American Conference on Biomedical Engineering, Florian\u00f3polis, Brazil."},{"key":"ref_60","unstructured":"Marimon, M., Gonzalez-Agirre, A., Intxaurrondo, A., Rodriguez, H., Martin, J.L., Villegas, M., and Krallinger, M. (2019, January 24). Automatic De-identification of Medical Texts in Spanish: The MEDDOCAN Track, Corpus, Guidelines, Methods and Evaluation of Results. Proceedings of the IberLEF@ SEPLN, Bilbao, Spain."},{"key":"ref_61","unstructured":"Berg, H., and Dalianis, H. (2020, January 11\u201316). A Semi-supervised Approach for De-identification of Swedish Clinical Text. Proceedings of the 12th Language Resources and Evaluation Conference, Marseille, France."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Ramshaw, L.A., and Marcus, M.P. (1999). Text chunking using transformation-based learning. Natural Language Processing Using Very Large Corpora, Springer.","DOI":"10.1007\/978-94-017-2390-9_10"},{"key":"ref_63","unstructured":"European Parliament and Council of the European Union (2024, November 05). Regulation (EU) 2016\/679 of the European Parliament and of the Council. Available online: https:\/\/data.europa.eu\/eli\/reg\/2016\/679\/oj."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"S20","DOI":"10.1016\/j.jbi.2015.07.020","article-title":"Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2\/UTHealth corpus","volume":"58","author":"Stubbs","year":"2015","journal-title":"J. Biomed. Inform."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Jantscher, M., Gunzer, F., Kern, R., Hassler, E., Tschauner, S., and Reishofer, G. (2023). Information extraction from German radiological reports for general clinical text and language understanding. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-29323-3"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., and Liu, T. (2024). A Survey on In-context Learning. arXiv.","DOI":"10.18653\/v1\/2024.emnlp-main.64"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N.A., Khashabi, D., and Hajishirzi, H. (2023, January 9\u201314). Self-Instruct: Aligning Language Models with Self-Generated Instructions. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada.","DOI":"10.18653\/v1\/2023.acl-long.754"},{"key":"ref_68","unstructured":"Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T.B. (2025, January 27). Stanford Alpaca: An Instruction-Following Llama Model. Available online: https:\/\/crfm.stanford.edu\/2023\/03\/13\/alpaca.html."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Lv, K., Yang, Y., Liu, T., Guo, Q., and Qiu, X. (2024, January 11\u201316). Full Parameter Fine-tuning for Large Language Models with Limited Resources. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.acl-long.445"},{"key":"ref_70","unstructured":"Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., and Gao, J. (2024). Large language models: A survey. arXiv."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Sun, C., Yang, Z., Wang, L., Zhang, Y., Lin, H., and Wang, J. (2021). Biomedical named entity recognition using BERT in the machine reading comprehension framework. J. Biomed. Inform., 118.","DOI":"10.1016\/j.jbi.2021.103799"},{"key":"ref_72","unstructured":"Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2020). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv."},{"key":"ref_73","unstructured":"Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozi\u00e8re, B., Goyal, N., Hambro, E., and Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv."},{"key":"ref_74","first-page":"1877","article-title":"Language Models are Few-Shot Learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_75","first-page":"1","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"Chung","year":"2024","journal-title":"J. Mach. Learn. Res."},{"key":"ref_76","unstructured":"OpenAI (2024, November 12). GPT-3.5 Turbo. Available online: https:\/\/platform.openai.com\/docs\/models\/gpt-3-5#gpt-3-5-turbo."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1038\/s41591-023-02552-9","article-title":"Optimized glycemic control of type 2 diabetes with reinforcement learning: A proof-of-concept trial","volume":"29","author":"Wang","year":"2023","journal-title":"Nat. Med."},{"key":"ref_78","unstructured":"OpenAI (2025, January 24). Hello GPT-4o. Available online: https:\/\/openai.com\/index\/hello-gpt-4o\/."},{"key":"ref_79","unstructured":"Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., and Fan, A. (2024). The llama 3 herd of models. arXiv."},{"key":"ref_80","unstructured":"Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv."},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3703155","article-title":"A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions","volume":"43","author":"Huang","year":"2023","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-017-1486-2","article-title":"Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers","volume":"18","author":"Eisinga","year":"2017","journal-title":"BMC Bioinform."},{"key":"ref_83","first-page":"0073","article-title":"On the Limitations of Zero-Shot Classification of Causal Relations by LLMs (Work in Progress)","volume":"1613","author":"Kanjirangat","year":"2024","journal-title":"Proc. ISSN"},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Gao, J., Lu, C., Ding, X., Li, Z., Liu, T., and Qin, B. (2024). Enhancing Complex Causality Extraction via Improved Subtask Interaction and Knowledge Fusion. arXiv.","DOI":"10.1007\/978-981-97-9440-9_6"},{"key":"ref_85","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_86","doi-asserted-by":"crossref","unstructured":"Ross, A., Willson, V.L., Ross, A., and Willson, V.L. (2017). Paired samples T-test. Basic and Advanced Statistical Tests: Writing Results Sections and Creating Tables and Figures, Sense Publishers.","DOI":"10.1007\/978-94-6351-086-8"},{"key":"ref_87","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv."},{"key":"ref_88","unstructured":"Bai, G., Chai, Z., Ling, C., Wang, S., Lu, J., Zhang, N., Shi, T., Yu, Z., Zhu, M., and Zhang, Y. (2024). Beyond efficiency: A systematic survey of resource-efficient large language models. arXiv."},{"key":"ref_89","unstructured":"Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., and Smith, N. (2020). Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s10462-024-10896-y","article-title":"Towards trustworthy LLMs: A review on debiasing and dehallucinating in large language models","volume":"57","author":"Lin","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"You, Z., Lee, H., Mishra, S., Jeoung, S., Mishra, A., Kim, J., and Diesner, J. (2024, January 16). Beyond Binary Gender Labels: Revealing Gender Bias in LLMs through Gender-Neutral Name Predictions. Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.gebnlp-1.16"},{"key":"ref_92","unstructured":"Halevi, S., and Rabin, T. (2006). Calibrating Noise to Sensitivity in Private Data Analysis. Proceedings of the Theory of Cryptography, Springer."},{"key":"ref_93","unstructured":"Kone\u010dn\u1ef3, J., McMahan, H.B., Yu, F.X., Richt\u00e1rik, P., Suresh, A.T., and Bacon, D. (2016, January 6\u20138). Federated learning: Strategies for improving communication efficiency. Proceedings of the NIPS Workshop on Private Multi-Party Machine Learning, Barcelona, Spain."},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3487890","article-title":"Defenses to membership inference attacks: A survey","volume":"56","author":"Hu","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., and Zhang, L. (2016, January 24\u201328). Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.","DOI":"10.1145\/2976749.2978318"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"He, Z., Zhang, T., and Lee, R.B. (2019, January 9\u201313). Model inversion attacks against collaborative inference. Proceedings of the 35th Annual Computer Security Applications Conference, San Juan, PR, USA.","DOI":"10.1145\/3359789.3359824"},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1109\/COMST.2019.2944748","article-title":"Differential privacy techniques for cyber physical systems: A survey","volume":"22","author":"Hassan","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"106775","DOI":"10.1016\/j.knosys.2021.106775","article-title":"A survey on federated learning","volume":"216","author":"Zhang","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.jnca.2016.11.027","article-title":"Cloud security issues and challenges: A survey","volume":"79","author":"Singh","year":"2017","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_100","unstructured":"Bagdasaryan, E., Poursaeed, O., and Shmatikov, V. (2019, January 8\u201314). Differential Privacy Has Disparate Impact on Model Accuracy. Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Vancouver, BC, Canada."},{"key":"ref_101","unstructured":"Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., and Shmatikov, V. (2020, January 26\u201328). How to backdoor federated learning. Proceedings of the International Conference on Artificial Intelligence and Statistics, Online."},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s42256-019-0055-y","article-title":"Establishing the rules for building trustworthy AI","volume":"1","author":"Floridi","year":"2019","journal-title":"Nat. Mach. Intell."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/2\/112\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:28:00Z","timestamp":1760027280000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/2\/112"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,6]]},"references-count":102,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["info16020112"],"URL":"https:\/\/doi.org\/10.3390\/info16020112","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,6]]}}}