{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T04:58:18Z","timestamp":1771477098640,"version":"3.50.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T00:00:00Z","timestamp":1740096000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T00:00:00Z","timestamp":1740096000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Drug Saf"],"published-print":{"date-parts":[[2025,6]]},"DOI":"10.1007\/s40264-025-01525-w","type":"journal-article","created":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:17:21Z","timestamp":1740147441000},"page":"667-677","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Effectiveness of Transformer-Based Large Language Models in Identifying Adverse Drug Reaction Relations from Unstructured Discharge Summaries in Singapore"],"prefix":"10.1007","volume":"48","author":[{"given":"Yen Ling","family":"Koon","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan Tung","family":"Lam","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui Xing","family":"Tan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Desmond Hwee Chun","family":"Teo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing Wei","family":"Neo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aaron Jun Yi","family":"Yap","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pei San","family":"Ang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Celine Ping Wei","family":"Loke","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mun Yee","family":"Tham","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siew Har","family":"Tan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sally Leng Bee","family":"Soh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Belinda Qin Pei","family":"Foo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zheng Jye","family":"Ling","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James Luen Wei","family":"Yip","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9613-6994","authenticated-orcid":false,"given":"Sreemanee Raaj","family":"Dorajoo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,2,21]]},"reference":[{"key":"1525_CR1","doi-asserted-by":"publisher","first-page":"204209862093859","DOI":"10.1177\/2042098620938595","volume":"11","author":"M Alomar","year":"2020","unstructured":"Alomar M, et al. Post marketing surveillance of suspected adverse drug reactions through spontaneous reporting: current status, challenges and the future. Ther Adv Drug Saf. 2020;11:2042098620938595.","journal-title":"Ther Adv Drug Saf"},{"issue":"Suppl 1","key":"1525_CR2","doi-asserted-by":"publisher","first-page":"S66","DOI":"10.4103\/0976-500X.120955","volume":"4","author":"C Palleria","year":"2013","unstructured":"Palleria C, et al. Limitations and obstacles of the spontaneous adverse drugs reactions reporting: two \u201cchallenging\u201d case reports. J Pharmacol Pharmacother. 2013;4(Suppl 1):S66-72.","journal-title":"J Pharmacol Pharmacother"},{"issue":"9","key":"1525_CR3","doi-asserted-by":"publisher","first-page":"965","DOI":"10.1002\/pds.4966","volume":"29","author":"M Khalili","year":"2020","unstructured":"Khalili M, et al. Interventions to improve adverse drug reaction reporting: a scoping review. Pharmacoepidemiol Drug Saf. 2020;29(9):965\u201392.","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"1525_CR4","doi-asserted-by":"publisher","DOI":"10.1093\/database\/baac071","author":"G Gonzalez-Hernandez","year":"2022","unstructured":"Gonzalez-Hernandez G, et al. Challenges and opportunities for mining adverse drug reactions: perspectives from pharma, regulatory agencies, healthcare providers and consumers. Database (Oxford). 2022. https:\/\/doi.org\/10.1093\/database\/baac071.","journal-title":"Database (Oxford)"},{"issue":"3","key":"1525_CR5","doi-asserted-by":"publisher","first-page":"1235","DOI":"10.1111\/bcp.15068","volume":"88","author":"A Wasylewicz","year":"2022","unstructured":"Wasylewicz A, et al. Identifying adverse drug reactions from free-text electronic hospital health record notes. Br J Clin Pharmacol. 2022;88(3):1235\u201345.","journal-title":"Br J Clin Pharmacol"},{"key":"1525_CR6","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.bdr.2016.04.001","volume":"4","author":"PS Ang","year":"2016","unstructured":"Ang PS, et al. Towards human-machine collaboration in creating an evaluation corpus for adverse drug events in discharge summaries of electronic medical records. Big Data Res. 2016;4:37\u201343.","journal-title":"Big Data Res"},{"issue":"1","key":"1525_CR7","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1007\/s40264-018-0762-z","volume":"42","author":"A Jagannatha","year":"2019","unstructured":"Jagannatha A, et al. Overview of the first natural language processing challenge for extracting medication, indication, and adverse drug events from electronic health record notes (MADE 1.0). Drug Saf. 2019;42(1):99\u2013111.","journal-title":"Drug Saf"},{"issue":"4","key":"1525_CR8","doi-asserted-by":"publisher","first-page":"813","DOI":"10.1093\/jamia\/ocw180","volume":"24","author":"A Cocos","year":"2017","unstructured":"Cocos A, Fiks AG, Masino AJ. Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts. J Am Med Inform Assoc. 2017;24(4):813\u201321.","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"1525_CR9","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1093\/jamia\/ocz101","volume":"27","author":"F Christopoulou","year":"2020","unstructured":"Christopoulou F, et al. Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods. J Am Med Inform Assoc. 2020;27(1):39\u201346.","journal-title":"J Am Med Inform Assoc"},{"issue":"8","key":"1525_CR10","doi-asserted-by":"publisher","first-page":"853","DOI":"10.1007\/s40264-022-01196-x","volume":"45","author":"HX Tan","year":"2022","unstructured":"Tan HX, et al. Combining machine learning with a rule-based algorithm to detect and identify related entities of documented adverse drug reactions on hospital discharge summaries. Drug Saf. 2022;45(8):853\u201362.","journal-title":"Drug Saf"},{"key":"1525_CR11","doi-asserted-by":"crossref","unstructured":"Kumar A, Makhija P, Gupta A. Noisy text data: Achilles\u2019 heel of BERT. In: Xu W, et al., editors. Online : Association for Computational Linguistics. 2020. pp. 16\u201321.","DOI":"10.18653\/v1\/2020.wnut-1.3"},{"issue":"1","key":"1525_CR12","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2022.103135","volume":"60","author":"X Chen","year":"2023","unstructured":"Chen X, et al. Dealing with textual noise for robust and effective BERT re-ranking. Inf Process Manag. 2023;60(1): 103135.","journal-title":"Inf Process Manag"},{"key":"1525_CR13","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AE Johnson","year":"2016","unstructured":"Johnson AE, et al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3: 160035.","journal-title":"Sci Data"},{"key":"1525_CR14","unstructured":"Johnson A, Pollard T, Mark R. MIMIC-III clinical database demo (version 1.4). In: PhysioNet. 2019."},{"key":"1525_CR15","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.ijmedinf.2019.04.017","volume":"128","author":"Y Tang","year":"2019","unstructured":"Tang Y, et al. Detecting adverse drug reactions in discharge summaries of electronic medical records using Readpeer. Int J Med Inform. 2019;128:62\u201370.","journal-title":"Int J Med Inform"},{"key":"1525_CR16","first-page":"2895","volume-title":"Matching the blanks: distributional similarity for relation learning","author":"L Baldini Soares","year":"2019","unstructured":"Baldini Soares L, et al. Matching the blanks: distributional similarity for relation learning. Association for Computational Linguistics: Florence; 2019. p. 2895\u2013905."},{"key":"1525_CR17","first-page":"4171","volume-title":"BERT: pre-training of deep bidirectional transformers for language understanding","author":"J Devlin","year":"2019","unstructured":"Devlin J, et al. BERT: pre-training of deep bidirectional transformers for language understanding. Association for Computational Linguistics: Minneapolis, Minnesota; 2019. p. 4171\u201386."},{"key":"1525_CR18","doi-asserted-by":"publisher","unstructured":"Clark K, et al. ELECTRA: pre-training text encoders as discriminators rather than generators. 2020. arXiv:2003.10555. https:\/\/doi.org\/10.48550\/arXiv.2003.10555.","DOI":"10.48550\/arXiv.2003.10555"},{"key":"1525_CR19","first-page":"8003","volume-title":"LinkBERT: pretraining language models with document links","author":"M Yasunaga","year":"2022","unstructured":"Yasunaga M, Leskovec J, Liang P. LinkBERT: pretraining language models with document links. Association for Computational Linguistics: Dublin; 2022. p. 8003\u201316."},{"key":"1525_CR20","unstructured":"Lan Z, et al. ALBERT: a Lite BERT for self-supervised learning of language representations. 2019. arXiv:1909.11942."},{"issue":"11","key":"1525_CR21","doi-asserted-by":"publisher","first-page":"332","DOI":"10.3390\/fi14110332","volume":"14","author":"X Li","year":"2022","unstructured":"Li X, Lei Y, Ji S. BERT- and BiLSTM-based sentiment analysis of online chinese buzzwords. Future Internet. 2022;14(11):332.","journal-title":"Future Internet"},{"issue":"2","key":"1525_CR22","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1055\/s-0038-1646963","volume":"9","author":"JC Smith","year":"2018","unstructured":"Smith JC, et al. Evaluation of a novel system to enhance clinicians\u2019 recognition of preadmission adverse drug reactions. Appl Clin Inform. 2018;9(2):313\u201325.","journal-title":"Appl Clin Inform"},{"key":"1525_CR23","doi-asserted-by":"publisher","unstructured":"Gu Y, et al. Domain-specific language model pretraining for biomedical natural language processing. 2020. arXiv:2007.15779. https:\/\/doi.org\/10.48550\/arXiv.2007.15779.","DOI":"10.48550\/arXiv.2007.15779"},{"key":"1525_CR24","doi-asserted-by":"publisher","unstructured":"Alsentzer E, et al. Publicly available clinical BERT embeddings. 2019. arXiv:1904.03323https:\/\/doi.org\/10.48550\/arXiv.1904.03323.","DOI":"10.48550\/arXiv.1904.03323"},{"key":"1525_CR25","doi-asserted-by":"crossref","unstructured":"Hessel, J. and A. Schofield, How effective is BERT without word ordering? Implications for language understanding and data privacy. Online: Association for Computational Linguistics; 2021. pp. 204\u2013211.","DOI":"10.18653\/v1\/2021.acl-short.27"},{"key":"1525_CR26","doi-asserted-by":"publisher","unstructured":"Sugawara S, et al. Assessing the benchmarking capacity of machine reading comprehension datasets. 2019. arXiv:1911.09241. https:\/\/doi.org\/10.48550\/arXiv.1911.09241.","DOI":"10.48550\/arXiv.1911.09241"},{"key":"1525_CR27","unstructured":"Gonz\u00e1lez-Carvajal S, Garrido-Merch\u00e1n EC. Comparing BERT against traditional machine learning text classification. 2020. arXiv:2005.13012."},{"key":"1525_CR28","doi-asserted-by":"crossref","unstructured":"Guven ZA. Comparison of BERT models and machine learning methods for sentiment analysis on Turkish Tweets. In: 2021 6th international conference on computer science and engineering (UBMK). 2021. pp. 98\u2013101.","DOI":"10.1109\/UBMK52708.2021.9559014"},{"issue":"3","key":"1525_CR29","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1038\/s42256-023-00626-4","volume":"5","author":"N Ding","year":"2023","unstructured":"Ding N, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nat Mach Intell. 2023;5(3):220\u201335.","journal-title":"Nat Mach Intell"},{"issue":"1","key":"1525_CR30","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1038\/s41746-022-00742-2","volume":"5","author":"X Yang","year":"2022","unstructured":"Yang X, et al. A large language model for electronic health records. npj Digit Med. 2022;5(1):194.","journal-title":"npj Digit Med"},{"issue":"1","key":"1525_CR31","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1007\/s40264-018-0763-y","volume":"42","author":"AB Chapman","year":"2019","unstructured":"Chapman AB, et al. Detecting adverse drug events with rapidly trained classification models. Drug Saf. 2019;42(1):147\u201356.","journal-title":"Drug Saf"},{"key":"1525_CR32","doi-asserted-by":"publisher","DOI":"10.3389\/fcimb.2020.572912","volume":"10","author":"J Ramirez","year":"2020","unstructured":"Ramirez J, et al. Antibiotics as major disruptors of gut microbiota. Front Cell Infect Microbiol. 2020;10: 572912.","journal-title":"Front Cell Infect Microbiol"},{"issue":"8010","key":"1525_CR33","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1038\/d41586-024-01215-0","volume":"629","author":"C Willyard","year":"2024","unstructured":"Willyard C. Do cutting-edge CAR-T-cell therapies cause cancer? What the data say. Nature. 2024;629(8010):22\u20134.","journal-title":"Nature"},{"issue":"12","key":"1525_CR34","doi-asserted-by":"publisher","DOI":"10.1002\/ccr3.5144","volume":"9","author":"VVK Matli","year":"2021","unstructured":"Matli VVK, et al. Normoglycemic diabetic ketoacidosis in a type 2 diabetes patient on dapagliflozin: a case report. Clin Case Rep. 2021;9(12): e05144.","journal-title":"Clin Case Rep"},{"issue":"1","key":"1525_CR35","doi-asserted-by":"publisher","first-page":"33","DOI":"10.4103\/0973-1229.87287","volume":"10","author":"A Nischal","year":"2012","unstructured":"Nischal A, et al. Suicide and antidepressants: what current evidence indicates. Mens Sana Monogr. 2012;10(1):33\u201344.","journal-title":"Mens Sana Monogr"},{"issue":"5","key":"1525_CR36","doi-asserted-by":"publisher","first-page":"844","DOI":"10.3390\/math10050844","volume":"10","author":"E Savini","year":"2022","unstructured":"Savini E, Caragea C. Intermediate-task transfer learning with BERT for sarcasm detection. Mathematics. 2022;10(5):844.","journal-title":"Mathematics"},{"key":"1525_CR37","doi-asserted-by":"crossref","unstructured":"Pruksachatkun Y, et al. Intermediate-task transfer learning with pretrained language models: when and why does it work? In: Annual meeting of the association for computational linguistics. 2020. pp. 5231\u20135247.","DOI":"10.18653\/v1\/2020.acl-main.467"}],"container-title":["Drug Safety"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40264-025-01525-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40264-025-01525-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40264-025-01525-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T13:48:15Z","timestamp":1747921695000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40264-025-01525-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,21]]},"references-count":37,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["1525"],"URL":"https:\/\/doi.org\/10.1007\/s40264-025-01525-w","relation":{},"ISSN":["0114-5916","1179-1942"],"issn-type":[{"value":"0114-5916","type":"print"},{"value":"1179-1942","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,21]]},"assertion":[{"value":"27 January 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 February 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was conducted under the SAPhIRE Project, funded by a Strategic Positioning Fund grant from the Biomedical Research Council of the Agency for Science, Technology and Research of Singapore (SPF2014\/001).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Funding"}},{"value":"The authors have no conflicts of interest that are directly relevant to the content of this article. The view expressed in this article are the authors\u2019 personal views and may not be understood or quoted as being made on behalf or reflect the positions of HSA, NUH and NUS.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"No ethics approval was required for this study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"All data were processed and analysed using Python 3.8. The code developed for this study can be made available for research purposes upon justified request. Interested researchers should contact the corresponding author via email, providing a concise outline of their intended use for the code.","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}},{"value":"The models developed in this study were trained on confidential clinical data. To protect patient privacy and prevent potential information leakage, the trained models cannot be made available.","order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Model availability"}},{"value":"The MIMIC-III database [, ], an open-source resource utilised in this study, is available through Google Cloud Platform (GCP) and Amazon Web Services (AWS). To facilitate access, investigators are required to append the appropriate cloud identifier to their PhysioNet profile. More information is provided on the official MIMIC-III website ().","order":9,"name":"Ethics","group":{"name":"EthicsHeading","label":"Availability of data and material"}},{"value":"YL Koon, YT Lam and SR Dorajoo designed the research and models and did the computational framework to analyse the data. PS Ang, CWP Loke, MY Tham, SH Tan, SBL Soh, BPQ Foo, DCH Teo and SR Dorajoo provided the domain expertise for manual annotation of discharge summaries and developed the list of trigger words, short forms and acronyms. HX Tan, DCH Teo, JW Neo, AJY Yap and PS Ang participated in discussions and refined the results. ZJ Ling and JWL Yip coordinated the provision of NUH data and participated in discussions. All authors read and approved the final version of this manuscript.","order":10,"name":"Ethics","group":{"name":"EthicsHeading","label":"Author contributions"}}]}}