{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T09:31:23Z","timestamp":1780392683564,"version":"3.54.1"},"reference-count":31,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T00:00:00Z","timestamp":1667865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, <jats:italic>in silico<\/jats:italic> and <jats:italic>in vitro<\/jats:italic> approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate safety concerns earlier in the drug development process. Although many AI tools are available, their acceptance in regulatory decision-making for drug efficacy and safety evaluation is still a challenge. It is a common perception that an AI model improves with more data, but does reality reflect this perception in drug safety assessments? Importantly, a model aiming at regulatory application needs to take a broad range of model characteristics into consideration. Among them is adaptability, defined as the adaptive behavior of a model as it is retrained on unseen data. This is an important model characteristic which should be considered in regulatory applications. In this study, we set up a comprehensive study to assess adaptability in AI by mimicking the real-world scenario of the annual addition of new drugs to the market, using a model we previously developed known as DeepDILI for predicting drug-induced liver injury (DILI) with a novel Deep Learning method. We found that the target test set plays a major role in assessing the adaptive behavior of our model. Our findings also indicated that adding more drugs to the training set does not significantly affect the predictive performance of our adaptive model. We concluded that the proposed adaptability assessment framework has utility in the evaluation of the performance of a model over time.<\/jats:p>","DOI":"10.3389\/frai.2022.1034631","type":"journal-article","created":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T06:34:35Z","timestamp":1667889275000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury"],"prefix":"10.3389","volume":"5","author":[{"given":"Skylar","family":"Connor","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ting","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruth","family":"Roberts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shraddha","family":"Thakkar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhichao","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weida","family":"Tong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2022,11,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/15353702211052280","article-title":"Emerging technologies and their impact on regulatory science","volume":"247","author":"Anklam","year":"2022","journal-title":"Exp. Biol. Med"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12864-019-6413-7","article-title":"The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation","volume":"21","author":"Chicco","year":"2020","journal-title":"BMC genomics."},{"key":"B3","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/s13040-021-00244-z","article-title":"The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation","volume":"14","author":"Chicco","year":"2021","journal-title":"BioData Min."},{"key":"B4","volume-title":"Advancing Regulatory Science at FDA: Focus Areas of Regulatory Science (FARS)","year":"2021"},{"key":"B5","first-page":"2280","article-title":"\u201cTools and Algorithms for the Construction and Analysis of Systems.\u201d","volume-title":"Lecture Notes in Computer Science","author":"Groce","year":"2002"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1007\/s11030-021-10217-3","article-title":"Artificial intelligence to deep learning: machine intelligence approach for drug discovery","volume":"25","author":"Gupta","year":"2021","journal-title":"Mol. Divers"},{"key":"B7","doi-asserted-by":"publisher","first-page":"987","DOI":"10.1126\/science.1204432","article-title":"Advancing regulatory science","volume":"331","author":"Hamburg","year":"2011","journal-title":"Science."},{"key":"B8","unstructured":"Software as a Medical Device (SaMD)2018"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1021\/ci800038f","article-title":"Mold2, molecular descriptors from 2D structures for chemoinformatics and toxicoinformatics","volume":"48","author":"Hong","year":"2008","journal-title":"J. Chem. Inf. Model."},{"key":"B10","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1021\/acs.chemrestox.0c00374","article-title":"DeepDILI: deep learning-powered drug-induced liver injury prediction using model-level representation","volume":"34","author":"Li","year":"2021","journal-title":"Chem. Res. Toxicol."},{"key":"B11","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1109\/TG.2018.2883661","article-title":"Toward personalized adaptive gamification: a machine learning model for predicting performance","volume":"12","author":"L\u00f3pez","year":"2018","journal-title":"IEEE Trans. Games."},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jcim.2c00765","article-title":"Investigation of a data split strategy involving the time axis in adverse event prediction using machine learning","author":"Morita","year":"2022","journal-title":"arXiv preprint arXiv:"},{"key":"B13","doi-asserted-by":"publisher","first-page":"S5","DOI":"10.1016\/j.kjms.2012.05.003","article-title":"Impact of regulatory science on global public health","volume":"28","author":"Patel","year":"2012","journal-title":"Kaohsiung J. Med. Sci"},{"key":"B14","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1001\/archinte.1965.03860140008003","article-title":"Drug-induced liver disease: a penalty for progress","volume":"115","author":"PoPPer","year":"1965","journal-title":"Arch. Intern. Med"},{"key":"B15","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1016\/j.yrtph.2012.02.001","article-title":"Advancing global health through regulatory science research: summary of the global summit on regulatory science research and innovation","volume":"62","author":"Slikker","year":"2012","journal-title":"Regul. Toxicol. Pharmacol"},{"key":"B16","year":"2021"},{"key":"B17","year":"2020"},{"key":"B18","year":"2022"},{"key":"B19","year":"2021"},{"key":"B20","year":"2022"},{"key":"B21","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1016\/j.drudis.2019.09.022","article-title":"Drug-induced liver injury severity and toxicity (DILIst): Binary classification of 1279 drugs by human hepatotoxicity","volume":"25","author":"Thakkar","year":"2020","journal-title":"Drug Discov"},{"key":"B22","volume-title":"FDA's Predictive Toxicology Roadmap","year":"2017"},{"key":"B23","volume-title":"International Medical Device Regulators Forum.","year":""},{"key":"B24","volume-title":"Proposed Regulatory Framework for Modifications to Artificial Intelligence\/Machine Learning (AI\/ML)-Based Software as a Medical Device","year":""},{"key":"B25","unstructured":"Drug Development Tool (DDT) Qualification Programs"},{"key":"B26","volume-title":"Artificial intelligence\/machine learning (AI\/ML)\u2013based software as a medical device (SaMD) action plan","year":""},{"key":"B27","unstructured":"Innovative Science and Technology Approaches for New Drugs (ISTAND) Pilot Program"},{"key":"B28","doi-asserted-by":"publisher","first-page":"845","DOI":"10.1016\/j.jacbts.2019.10.008","article-title":"Limitations of animal studies for predicting toxicity in clinical trials: is it time to rethink our current approach?","volume":"4","author":"Van Norman","year":"2019","journal-title":"JACC: Basic Transl. Sci"},{"key":"B29","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1109\/ASPDAC.2016.7428100","article-title":"\u201cNovel applications of deep learning hidden features for adaptive testing.\u201d","volume-title":"2016 21st Asia and South Pacific Design Automation Conference (ASP-DAC)","author":"Xiao","year":"2016"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1250","DOI":"10.1016\/j.enbuild.2005.02.005","article-title":"On-line building energy prediction using adaptive artificial neural networks","volume":"37","author":"Yang","year":"2005","journal-title":"Energy Build"},{"key":"B31","volume-title":"Hepatotoxicity: the adverse effects of drugs and other chemicals on the liver","author":"Zimmerman","year":"1999"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2022.1034631\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T06:34:47Z","timestamp":1667889287000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2022.1034631\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,8]]},"references-count":31,"alternative-id":["10.3389\/frai.2022.1034631"],"URL":"https:\/\/doi.org\/10.3389\/frai.2022.1034631","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,8]]},"article-number":"1034631"}}